This page has been archived on the Web

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Evaluation of Waste Reduction and Management

February 2012

Report Clearance Steps

Planning phase completedFebruary 2011
Report sent for management responseOctober 2011
Management response receivedNovember 2011
Report completedNovember 2011
Report approved by Departmental Evaluation Committee (DEC)February 2012

 

Acronyms used in the report

AAFCAgriculture and Agri-Food Canada
AANDCAboriginal Affairs and Northern Development Canada
APSDAboriginal and Public Sectors Division
AWWGArctic Wastewater Working Group
CBCCanadian Broadcasting Corporation
CCMECanadian Council of Ministers of the Environment
CEPA 1999Canadian Environmental Protection Act, 1999
CESDCommissioner of the Environment and Sustainable Development
CIOBChief Information Officer Branch
CMPChemicals Management Plan
CNMTSCanadian Notice and Manifest Tracking System
CSBCorporate Services Branch
DFODepartment of Fisheries and Oceans
DPRDepartmental Performance Report
ECEnvironment Canada
EEMEnvironmental Effects Monitoring
EIHWHRMRExport and Import of Hazardous Waste and Hazardous Recyclable Material Regulations
ENGOEnvironmental non-governmental organization
EPODEnvironmental Protection Operations Directorate
EPRExtended Producer Responsibility
ESBEnvironmental Stewardship Branch
FAADForestry, Agriculture and Aquaculture Division
FAFisheries Act
FA-PPPFisheries Act-Pollution Prevention Provisions
FIRSTSFederal Identification Registry for Storage Tank Systems
FTEFull-time equivalent
G&CsGrants and contributions
LRADLegislative and Regulatory Affairs Directorate
MMERMetal Mining Effluent Regulations
MOUMemorandum of Understanding
MPDMining and Processing Division
MPMOMajor Projects Management Office
NAESINational Agri-Environmental Standards Initiative
NEEMONational Environmental Effects Monitoring Office
NEMISISNational Emergencies and Enforcement Information System and Intelligence System
NGONon-governmental organization
NPRINational Pollutant Release Inventory
O&MOperations and maintenance
OECDOrganisation for Economic Co-operation and Development
OGDOther government department
PAAProgram Activity Architecture
PCBPolychlorinated Biphenyl
PMEPPerformance measurement and evaluation plan
PPERPulp and Paper Effluent Regulations
PRSDPublic Resources and Sectors Directorate
RIASRegulatory Impact Analysis Statement
RISSRegulatory Information Submissions System
RMAFResults-Based Management and Accountability Framework
S&TScience & technology
SASub-activity
SSASub-sub-activity
SWMDSustainable Water Management Division
TIATailings impoundment area
TWRTailings waste rock
WRMWaste Reduction and Management
WRMDWaste Reduction and Management Division
WSERWastewater Systems Effluent Regulations

 

Acknowledgements

The Evaluation Project Team would like to thank those individuals who contributed to this project, particularly members of the Evaluation Committee as well as all interviewees and survey respondents who provided insights and comments crucial to this evaluation.

The Evaluation Project Team was led by Shelley Tice, under the direction of the Environment Canada Evaluation Director, William Blois, and included Susan Wharton and Robert Tkaczyk.

Prepared by the Evaluation Division, Audit and Evaluation Branch

Table of Contents

Executive Summary

Introduction

This report presents the results of the Evaluation of Waste Reduction and Management conducted by Environment Canada’s Audit and Evaluation Branch between November 2010 and September 2011. The objective of the evaluation was to assess the relevance and performance (including effectiveness, efficiency and economy) of the activities which fall under the department’s Program Activity Architecture sub-sub-activity (SSA) 3.1.2.1, Waste Reduction and Management (WRM). The evaluation covered the four-year time frame from 2007-2008 to 2010-2011, while also addressing significant developments which occurred in the first quarter of 2011-2012.

Background

The activities of Environment Canada’s WRM SSA respond to obligations under federal legislation, such as the Fisheries Act(FA) and the Canadian Environmental Protection Act, 1999 (CEPA 1999), as well as international conventions. A primary driver for much of the work of the WRM SSA is subsection 36(3) of the Fisheries Act, which prohibits the deposit of deleterious substances into water frequented by fish, unless authorized by regulations. Requirements under CEPA 1999 related to the management of toxic substances and the management of hazardous and non-hazardous waste are also major drivers. While much of the WRM activity is focused on water pollution prevention, some activities, such as the management of hazardous wastes, include prevention of releases to other media. As such, the WRM SSA develops, implements and administers strategies, instruments and programs in order to manage the risks to the environment and human health from:

  • water pollution from a range of sectors, including forest products, mining and agriculture, related to the industrial production phase of materials and products to their end-of-life treatment/disposal and movement;
  • the deposit of wastewater effluent to water from wastewater systems (municipal, federal, Aboriginal and private); and
  • hazardous solid wastes, including the international and interprovincial movement and disposal of specific hazardous wastes and hazardous recyclable materials (stemming from municipal solid waste and elsewhere), as per international conventions to which Canada is a signatory.

The WRM SSA has annual expenditures ranging from $10.9 million in 2007-2008 to $13.2 million in 2010-2011.

Organizationally, the work of Environment Canada’s WRM SSA is conducted within the Environmental Stewardship Branch’s (ESB’s) Public Resources and Sectors Directorate (PRSD), which is composed of the following four divisions:

  • Mining and Processing Division (MPD);
  • Forestry, Agriculture and Aquaculture Division (FAAD);
  • Sustainable Water Management Division (SWMD), including responsibilities for wastewater; and
  • Waste Reduction and Management Division (WRMD), including environmental management of federal land responsibilities (also known as “Federal House”).

As the work of the WRM SSA is heavily regulatory, the following table identifies the relevant regulations linked to each of the four divisions. Given the regulatory nature of the SSA, the divisions work closely with the staff of the Compliance Promotion and Analysis Division and the Enforcement Branch, as reflected in PAA element 3.3.

Regulations Relevant to Waste Reduction and Management Activities
Effective DateMining and Processing Division
2002Metal Mining Effluent RegulationsMMER (FA)
 Forestry, Agriculture and Aquaculture Division
1992Pulp and Paper Effluent RegulationsPPER(FA)
1992Pulp and Paper Mill Defoamer and Wood Chip Regulations (CEPA)
1992Pulp and Paper Mill Effluent Chlorinated Dioxins and Furans Regulations (CEPA)
 Sustainable Water Management Division
ProposedProposed Wastewater System Effluent Regulations-WSER(FA)
 Waste Reduction and Management Division – Environmental management on federal land (Federal House)
2008Storage Tank Systems for Petroleum Products and Allied Petroleum Products Regulations (CEPA)
 Waste Reduction and Management Division – Storage, disposal and use of harmful substances
2008PCB Regulations (CEPA)
1996PCB Waste Export Regulations (CEPA)
1990Federal Mobile PCB Treatment and Destruction Regulations (CEPA)
ProposedProposed Regulation for Mercury-Containing Products (Regulations Respecting Products Containing Certain Substances Listed in Schedule 1 to CEPA)
 Waste Reduction and Management Division – Transboundary movement of hazardous waste and hazardous recyclable materials
2005Export and Import of Hazardous Waste and Hazardous Recyclable Material Regulations-EIHWHRMR (CEPA)
2002Interprovincial Movement of Hazardous Waste Regulations (CEPA)

Environment Canada also works with many stakeholders, as responsibilities for the management of waste and wastewater are shared across the federal, provincial/territorial and municipal levels. At the federal level, Environment Canada works primarily with the Department of Fisheries and Oceans for issues related to the Fisheries Act and with Aboriginal Affairs and Northern Development Canada for issues on Aboriginal lands, among other government departments. In many cases, provincial or territorial waste reduction regulations also exist, and Environment Canada works closely with provincial, territorial and municipal governments for consultation and national coordination.

Methodology

Three key considerations played a role in the design of the approach and methodology used for this evaluation.

This evaluation was one of the first evaluations conducted by Environment Canada’s Audit and Evaluation Branch addressing an evaluation at the Program Activity Architecture level. The WRM SSA cut across four divisions addressing different regulations, stakeholders and outcomes, and as such, the evaluation methodology was designed to ensure that both the data collection and analysis examined information at the division level, as well as at the overall Program Activity Architecture level. Upon completion of the evaluation an exercise was undertaken to reflect upon the experience, and to identify lessons learned which can be applied to similar evaluations in the future.

Second, other recent audits and evaluations have been conducted which offered insight into the WRM SSA. This evaluation builds upon these previous audits and evaluations, including relying on recent evidence as presented in these studies’ reports, to the degree that it was appropriate to do so.

Finally, the evaluation took extra efforts to ensure that the perspective of the regulated community was captured, because ultimately the success of these regulations is determined by the degree to which the regulations are understood and implemented.

With these considerations in mind, the following methods were used in the conduct of this evaluation, ensuring that the evaluation questions were addressed by multiple lines of evidence: a document review, a literature review, an administrative file review, key informant interviews with program staff representing the four divisions, other government and external stakeholders, and an online survey of external stakeholders and regulatees, environmental non-governmental organizations and other government stakeholders.

Findings and Conclusions

The following highlights the overall findings and conclusions of the evaluation.

Relevance

There is a continued need for the activities in support of Waste Reduction and Management at Environment Canada. The work of the SSA aligns with government priorities, the strategic outcomes of the Department and the legislated mandate of the federal government.

Activities of WRM contribute to the general health and safety of Canadians as well as to environmental health and ecosystem resilience in Canada through the provision of key regulations and non-regulatory instruments in the area of water quality and waste.

There is a clear legislated mandate for federal government involvement in the provision of waste reduction and management services, set by regulations and guidelines under CEPA 1999 and the Fisheries Act, as well as within the responsibilities of the Canadian Water Act. Additionally, the work of this SSA supports Canada’s commitment to the Basel Convention on the Control of Transboundary Movements of Hazardous Wastes and their Disposal.

Further, the activities of the WRM SSA provide support to the following recently articulated federal government priorities:

  • water quality and a reduction of risks associated with wastewater effluents;
  • Northern strategy focusing on pollution prevention and waste management activities in the North; and
  • clean drinking water and effective wastewater management on First Nations reserves.
Performance

Progress has been made towards the achievement of the Waste Reduction and Management SSA’s immediate and intermediate outcomes,[1] including the achievement of generally high compliance with WRM regulations (with the exception of the PCB Regulations), thereby contributing to reduced threats to the environment from deleterious and harmful substances.

For the most part, regulatees are aware of and understand the requirements of the Fisheries Act-Pollution Prevention Provisions and CEPA 1999, and Environment Canada’s WRM activities have contributed to progress in terms of regulatees meeting monitoring and reporting requirements for the management of harmful substances. Lower results were, however, identified in terms of the clarity of monitoring and reporting requirements and the degree to which regulations are well communicated and clearly understood, for the two regulations managed by the Waste Reduction and Management Division (WRMD) pertaining to the transboundary movement of hazardous waste, and also for the proposed Wastewater System Effluent Regulations (WSER), recognizing that some lack of clarity regarding the WSER is to be expected given that this regulation is still under development.

The data reveal generally high compliance results across all regulations (typically greater than 90%), with the exception of the PCB Regulations for which compliance results are estimated to be less than 50%. Multiple sources are available which speak to compliance, but there is variation in terms of how compliance results are calculated and reported among these sources. Attention should be given to clarify the methods by which calculations are based and the different subsets of regulatees they represent. Sources of compliance results include data from reporting databases (e.g., storage tanks, PCBs, Regulatory Information Submissions System [RISS]), the Enforcement Branch’s National Emergencies and Enforcement Information System and Intelligence System (NEMISIS), and from the Compliance Promotion and Analysis Division. While compliance results are generally strong, the fact that some regulatees lack the infrastructure resources required to comply with regulations (e.g., Aboriginal communities for the Storage Tank Systems for Petroleum Products and Allied Petroleum Products Regulations [hereafter referred to as the storage tank regulations]) is a barrier that impacts compliance results.

Those in the regulated community who had participated in compliance promotion and information sessions were very positive in terms of the usefulness of compliance promotion, as well as the quality of information provided. The evaluation also found a clear link between compliance results and compliance promotion activities. While the strategy of an intensive two‑year focus on compliance promotion for new or revised regulations was effective in most cases, this strategy does not work as well for those regulations where there is high turnover among regulatees, (notably for regulatees on Aboriginal lands), as is the case with the storage tank regulations and is also anticipated to be an issue for the WSER.

There was generally high recognition of the contribution of Environment Canada toward the achievement of the SSA’s wider environmental outcomes and stakeholders indicated that progress continues to be made in this area. Specific technical issues were identified, however, which were seen as limiting progress, including:

  • the acute lethality test linked to the PPER, which industry feels requires modification to allow for more efficient use of water (although the Forestry, Agriculture and Aquaculture Division (FAAD) is currently working with industry to examine the validity of their claims and to review potential options); and
  • tailings impoundment areas allowed under the MMER, as environmental non-governmental organizations question whether environmental impacts are being reduced as a result of this practice (although the Mining and Processing Division (MPD) notes that tailings impoundment areas are only authorized after it has been proven during an environmental assessment that the tailings impoundment areas is the best overall alternative).

Governance and priority-setting mechanisms for the SSA are appropriate and support the efficient delivery of the Waste Reduction and Management SSA’s outputs, although a lack of clarity in the management processes with respect to resource allocation exists for the Waste Reduction and Management Division (WRMD) and there appears to be limited engagement in the priority-setting process for both the Sustainable Water Management Division’s (SWMD’s) wastewater section and WRMD.  

Divisional planning and priority setting guide the activities of the Public Resources and Sectors Directorate divisions, and the appropriate processes are in place to coordinate with priority setting for the Compliance Promotion and Analysis Division and the Enforcement Branch. Concerns were identified with respect to a lack of clarity in the management processes in WRMD, specifically in the area of resource allocation; organizational changes and management turnover were identified as contributing to challenges in this area. Further, concerns were identified regarding the level of engagement in the priority setting for both SWMD’s wastewater section and WRMD, with a general sense that priority setting was top-down.

A lack of clarity with respect to the division of roles and responsibilities with the Department of Fisheries and Oceans for delivery of the Fisheries Act-Pollution Prevention Provisions, as well as some jurisdictional questions with the provinces, were identified as issues for the SSA.

The evaluation identified that changes are required to clarify responsibilities related to the Fisheries Act-Pollution Prevention Provisions and various jurisdictions involved, as evidenced by the following:

  • the approach to the application and reporting of this Act varies between Environment Canada and the Department of Fisheries and Oceans, and there are instances when individuals are referred back and forth between the two departments when requesting information;
  • ambiguity associated with the roles and responsibilities related to aquaculture;
  • survey respondents who contribute to the delivery of risk-management instruments/initiatives provided relatively low rankings with regard to how well roles and responsibilities are defined and understood among the various organizations/jurisdictions involved; and
  • jurisdictional questions regarding the role of Environment Canada versus the provinces were raised in a number of areas, including by external stakeholders for SWMD’s wastewater section for the WSER, and by both external and government stakeholders for WMRD regarding the Export and Import of Hazardous Waste and Hazardous Recyclable Material Regulations (EIHWHRMR).

The issue regarding lack of clarity with respect to delivery of the Fisheries Act-Pollution Prevention Provisions was also identified in the Commissioner of the Environment and Sustainable Development (CESD) spring 2009 audit report. This report identified concerns with the roles and responsibilities between Environment Canada and the Department of Fisheries and Oceans, and recommended the renewal of a dated Memorandum of Understanding (MOU) (1985) between the two departments, which aims to lay out the mutual responsibilities with respect to the administration and enforcement of the Fisheries Act-Pollution Prevention Provisions. The current evaluation also identified, through stakeholder interviews and document review, the outdated MOU between the Department of Fisheries and Oceans and Environment Canada as a contributing factor to this lack of clarity.

Extensive performance information coming from a variety of sources (e.g., the Compliance Promotion and Analysis Division and the Enforcement Branch, regulatory databases) exists; however, there is no cohesive performance story that speaks to the overall effectiveness of Environment Canada’s Waste Reduction and Management SSA and its regulations in achieving their environmental outcomes. Further, while a great deal of compliance information is available, it is often difficult to interpret what each of the individual sources represents and much of the data continues to be managed manually.

As monitoring and reporting is a requirement under most regulations, a considerable amount of performance information exists. However, some external stakeholders stated that they felt there has been a recent deterioration in the quality and quantity of data collected.

Compliance results are available from a number of data sources, including regulatory databases (e.g., Federal Identification Registry of Storage Tank Systems’ database [for storage tanks], the Canadian Notice and Movement Tracking System database [for the EIHWHRMR], the PCB database, and the Regulatory Information Submissions System [for the PPER and MMER], targeted enforcement activities reflected in the NEMISIS database, and information from the Compliance Promotion and Analysis Division. This information is presented in a variety of different reports (e.g., Fisheries Act Annual Report, summary reviews and reports related to the MMER and PPER, NEMISIS analyses). As each report represents compliance results calculated based on a different subset of regulatees, which is not clearly referenced, it is difficult to obtain the overall compliance story for a given regulation.

The WRM SSA is essentially divided into two areas: i) water quality (FAAD, MPD, SWMD) and ii) waste, including toxic substances (WRMD). While the three divisions that focus on water quality issues generally believe they have adequate information, WRMD identified that performance data are insufficient for decision making. Further, the two areas have different outcomes and use different language to describe their work, making it difficult to present a cohesive performance story across the entire SSA. Additionally, it may also be valuable to explore the opportunity to connect the Fisheries Act and related water quality work to the Department's broader water quality agenda.

The identification by external stakeholders that there has been a recent reduction in data quality and quantity may be due in part to the removal of the Environmental Effects Monitoring Program’s former online reporting systems in 2010, which occurred because it was not compliant with the Treasury Board Secretariat’s Common Look and Feel for the Internet Standards. Issues also exist with the Federal Identification Registry of Storage Tank Systems database’s (used for the storage tank regulations) online functionality, Canadian Notice and Movement Tracking System (for the EIHWHRMR) and the database for the PCB Regulations. While plans to address these issues have been identified, there is currently considerable variability in terms of the format in which reporting now takes place, with a mix of electronic and manual paper-based reporting.

Grants and contributions (G&Cs) are an exception to the relatively positive performance measurement story. While relatively few G&Cs exist within the SSA (between $37,000 and $90,000 per year), there was a lack of available performance information for them, resulting in an inability to report on the G&Cs’ overall performance relative to the SSA outcomes.

Limited financial information was available to support an analysis of efficiency and economy for the Waste Reduction and Management regulations.

In spite of the seemingly high compliance results for the regulations assessed within this SSA, the evaluation encountered difficulty determining the efficiency and economy of the regulations given the challenges associated with tracking/measuring resource allocations by individual regulation, including quantifying the level of support to the various regulations from departmental support functions (e.g., compliance promotion, enforcement), the regions (e.g., Environmental Protection Operations Division [EPOD]), and within PRSD divisions. Another element which contributed to this difficulty was the lack of budgeted financial information at the divisional level to compare forecasted versus actual expenditures for the PRSD divisions.

Interviews and documentation suggest that resource shortages constitute a challenge to realizing Environment Canada’s Waste Reduction and Management activities. Although the divisions are managing with their current levels through prioritization, several key activities are not taking place.

Evaluation evidence suggests that resource shortfalls are impacting the delivery of key activities. For example, resource shortages were both documented and identified by stakeholders as impeding the SSA’s ability to deliver on management actions stemming from the CESD spring 2009 audit, including reviewing and addressing several dated regulations, and clarifying roles and responsibilities related to the Fisheries Act-Pollution Prevention Provisions. They were also identified by stakeholders as having an impact on the delivery of key activities, including replacing manual reporting systems with online capability, meeting the high regulatory demand in the regions associated with environmental assessments for MPD, and the ability of WRMD to meet demand for export notifications in a timely manner. Furthermore, resources were flagged by stakeholders as an area of concern in terms of the Department’s ability to fund compliance promotion at levels needed to sustain high compliance results, on an ongoing basis, for the storage tank regulations, as well as for the WSER, post- implementation.

Recommendations

The following recommendations are based on the findings and conclusions of the evaluation. The evaluation recommendations are directed to the Assistant Deputy Minister (ADM)-Environmental Stewardship Branch (ESB), in light of the ADM’s responsibility for the overall management of this area.

Recommendation 1: Revise and update the management actions as committed to in response to the CESD spring 2009 audit recommendations, to ensure that they can be implemented.

The evaluation independently replicated many of the conclusions drawn by the CESD. In particular, the evaluation identified a need for high-level performance measurement data, a review of dated regulations, and greater clarity of roles and responsibilities between Environment Canada and the Department of Fisheries and Oceans related to the Fisheries Act-Pollution Prevention Provisions. These findings echo CESD findings related to identifying and tracking performance measurement information (1.93), the review of several dated regulations and guidelines (1.120), and clarification of roles and responsibilities between Department of Fisheries and Oceans and Environment Canada related to the Fisheries Act-Pollution Prevention Provisions (1.134). Environment Canada accepted the recommendations of the 2009 CSE Dreport and identified actions to address the issues. While Environment Canada has made some progress, commitments have not been met within identified timelines.

Management Response – Recommendation 1
ESB agrees with this recommendation. Environment Canada accepted the recommendations of the CESD report and developed a “CESD Management Action Plan” to address the issues (attached as Annex 3). While Environment Canada has made some progress, limited resources and the need for ongoing broader discussions between Environment Canada and the Department of Fisheries and Oceans relevant to delivering on Environment Canada’s CESD commitments have meant that the Department has not delivered all of the milestones within the timelines committed to in response to the May 2009 CESD report.
Management Action
ESB has a management action plan to deliver on its response to the CESD audit, is implementing the plan, and has a process to report on progress. EC’s CESD Management Action Plan Spring 2011 Update is attached as Annex 3.
TimelineDeliverable(s)Responsible Party

Recommendation 2: Conduct a review and develop action plans to strengthen internal Waste Reduction and Management Division (WRMD) processes related to improving the clarity and communication of:

WRMD priority setting and resource allocation decisions;

  1. WRMD priority setting and resource allocation decisions;
  2. reporting requirements for both the EIHWHRMR and the Interprovincial Movement of Hazardous Waste Regulations; and
  3. roles and responsibilities among various jurisdictions and organizations involved in the EIHWHRMR and the PCB Regulations.

While WRMD represents an important component of the WRM SSA, the division appears to face a number of challenges impacting its ability to achieve outcomes. Many of these issues relate to a need for more clearly defined processes and improved communication to stakeholders, including in relation to internal operations, regulatory reporting requirements, and roles and responsibilities across jurisdictions. 

WRMD priority-setting processes were identified as an area for improvement, as staff believed that engagement at the working level was limited and that resource allocation decisions were not clear. 

External communications were also identified as a challenge for WRMD. Evidence identified a lack of clarity both in terms of the communication of the EIHWHRMR and the Interprovincial Movement of Hazardous Waste Regulations and their associated reporting requirements. Program staff noted that in order for international commitments to be met, regulatees need to clearly understand reporting requirements. 

Finally, external stakeholders expressed a desire for greater harmonization with the provinces, both in relation to the EIHWHRMR and the PCB Regulations, and believed that a lack of consistency among jurisdictional requirements was hampering the achievement of environmental results of the EIHWHRMR.

Management Response – Recommendation 2

i. WRMD priority setting and resource allocation decisions

ESB agrees with this recommendation, but does not see it as a high priority item relative to other issues the program needs to address. Priority setting and related resource allocation decisions are often difficult to fully comprehend, particularly at the working level, given the multitude of competing considerations and changing circumstances. In order to improve transparency of these decisions within WRMD, quarterly division meetings will be instituted. A standing item on the agenda for these meetings will be resource allocations to the various files managed by the division.

ii. Reporting requirements for both the EIHWHRMR and the Interprovincial Movement of Hazardous Waste Regulations

ESB agrees with this recommendation. WRMD already puts considerable emphasis on communicating regulatory obligations to affected parties. Informal discussions occur daily between the division’s notification and manifest officers and regulatees on specific permit and movement issues. In addition, over the past two years WRMD has published a number of training and outreach documents on the Department’s website. The program will improve the clarity of reporting requirements and will reduce the administrative burden associated with those requirements through two planned initiatives: the planned amendments to the regulations and, most significantly, the work underway to move the Canadian Notice and Manifest Tracking System (CNMTS) to an online application and reporting system.

iii. Roles and responsibilities among various jurisdictions and organizations involved in the EIHWHRMR and the PCB Regulations.

ESB agrees with this recommendation. WRMD will address this recommendation through the planned amendments to consolidate these two regulations together with the Interprovincial Movement of Hazardous Waste Regulations. Among other things, the proposed amendments are expected to:

  • reduce the administrative burden imposed on regulatees, provinces and territories;
  • modify the regulatory reporting requirements to be compatible with the electronic submission of information; and
  • clarify the definitions of hazardous waste and hazardous recyclable material, to improve enforceability.
Management Action
To address the three elements of the recommendation, ESB proposes the three actions outlined below.
TimelineDeliverable(s)Responsible Party
ImmediateTo respond to Recommendation 2i, WRMD will hold quarterly division meetings to improve the transparency of resource allocation decisions.Director, WRMD
Ongoing, with expected completion date of March 31, 2014To respond to Recommendation 2ii, the program will put the Canadian Notice and Manifest Tracking System (CNMTS) online.Director, WRMD, contingent on resources and delivery by CSB
Ongoing, with expected completion date of December 31, 2012To implement Recommendation 2iii, the program will propose amendments to the three waste regulations, which are expected to improve the implementation and administration of these regulations, including clarifying roles.Director WRMD

Recommendation 3: Ensure performance measurement data are being collected and are readily available to communicate the performance story and support decision making by: i) developing and populating a performance measurement strategy for the SSA (or for its two sub-components); and ii) ensuring that all online reporting and data collection tools are fully functional.

Little strategic performance information is available that speaks to the overall effectiveness of the regulations or the SSA as a whole in achieving the environmental outcomes of the SSA and higher level sub-activity, although a great deal of performance information is being collected at the regulation level.

Work has begun in the form of separate logic models for the Fisheries Act-Pollution Prevention Provisions and a component of WRMD’s responsibilities, the waste reduction and management program, as well as through the development of a logic model for this evaluation. However, a coordinated approach to measuring against these outcomes is not yet occurring. In addition, performance measurement and evaluation plans are developed for all regulations that triage high (e.g., WSER). The performance measurement strategy should build upon and consolidate these existing strategies and should address the SSA’s components, water quality and waste, including toxic substances. In doing this it would address multiple performance measurement requirements, reduce duplication and reporting burden, and respond to departmental requirements. It would also include provisions to ensure appropriate performance information is collected for G&Cs spending.

There is a great deal of variability in terms of the format in which reporting takes place, with a mix of electronic and paper-based reporting. In order to ensure data are readily available, the SSA should work with the Information Management Directorate[2] within the Corporate Services Branch (CSB) to ensure all online reporting and data collection tools (e.g., Environmental Effects Monitoring Program, Federal Identification Registry of Storage Tank Systems, CNMTS, PCB database) are fully functional. This will also improve the consistency and efficiency with which monitoring and reporting takes place.

Management Response – Recommendation 3

ESB agrees with this recommendation. After the period on which this evaluation focused, the directorate (PRSD) contributed refinements related to the desired outcomes and appropriate indicators for this SSA as part of the Department’s recent efforts to improve its departmental Performance Measurement Framework for 2012-2013. PRSD will continue to support the Department’s ongoing work to further refine the Performance Measurement Framework, including for the WRM SSA.

PRSD also recognizes the need to develop performance measurement strategies for the two sub-components of SSA 3.1.2.1, as recommended by the evaluation:

  • The performance measurement strategy for the Fisheries Act sub-component will be the Results-Based Management and Accountability Framework (RMAF), which is a key deliverable in the Management Action Plan developed in response to the 2009 CESD audit (see Annex 3).
  • The waste sub-component will develop a performance measurement strategy that is built on the program’s existing Logic Model and its quarterly and annual reports on permitting activity and transboundary movements.

With respect to ensuring that performance measurement data are collected and used to communicate performance measurement, the program’s top priority is to complete the ongoing work to develop online reporting and data collection tools. These tools will provide information that will help PRSD to communicate performance and that will support informed decision making. Progress in this area will be contingent upon the continued availability of funding and departmental capacity to develop and implement these tools. As such, no time-bound commitment will be made to develop and implement these tools at this time.

Management Action
To address the recommendation, ESB proposes the three actions outlined below.
TimelineDeliverable(s)Responsible Party
March 2012The Performance Measurement Strategy for the Fisheries Act sub-component of the 3.1.2.1 SSA is the RMAF outlined in the CESD Management Action Plan (see Annex 3).Director, FAAD
June 30, 2012The Performance Measurement Strategy for the Waste Reduction and Management sub-component of the 3.1.2.1 built on the existing program Logic Model.Director, WRMD
OngoingWork with Corporate Services Branch to improve the functionality of online reporting and data collection tools.Director of WRMD for CNMTS, Federal Identification Registry for Storage Tank Systems (FIRSTS) and PCB Database, and other PRSD directors for shared databases, such as the Regulatory Information Submissions System and EEM. Contingent on resources and delivery by CSB.

 

1.0 Introduction

In 2010-2011 Environment Canada’s (EC’s) Evaluation Division, Audit and Evaluation Branch, conducted the Evaluation of the Waste Reduction and Management sub-sub-activity (SSA 3.1.2.1). An SSA-level evaluation represents a pilot approach to evaluation in order to better respond to the requirements of the new Policy on Evaluation to evaluate 100% of the Department’s direct program spending every five years. This evaluation covered the four-year time frame from 2007-2008 to 2010-2011, while also addressing significant developments which occurred in the first quarter of 2011-2012.

This document presents the findings and recommendations of the Evaluation of Waste Reduction and Management and is organized in the following way: Section 2.0 provides background information on the Program Activity Architecture (PAA) element; Section 3.0 presents the evaluation design, including the purpose and scope of the evaluation, as well as the methods used to conduct the evaluation, the limitations and lessons learned; Section 4.0 presents the evaluation’s findings; Sections 5.0 and 6.0 lay out, respectively, the conclusions and recommendations; and the management response is presented in Section 7.0.

2.0 Background

2.1 Program Profile

Work within this SSA falls under two key activities: the risk management of threats to water quality, and the management of hazardous waste and recycled materials, including the control of toxic substances.

Responsibilities for the management of waste and wastewater are shared across the federal, provincial/territorial and municipal levels. Provincial/territorial and municipal governments have responsibility over the permitting and day-to-day management of hazardous waste and hazardous recyclable materials, municipal solid waste and wastewater on non-federal and non-Aboriginal lands, as well as the intra-provincial/territorial movement of hazardous waste and hazardous recyclable materials. Provinces and territories also have responsibilities for permitting of sector activities (mining and processing facilities, including the management of their waste [waste rock, tailings, effluents]; agriculture; and pulp and paper). The federal government plays a role in managing water quality, with a focus on EC’s mandate to preserve and enhance the quality of Canada’s natural environment, including action to protect and enhance water quality through scientific research, monitoring risk assessment and risk management.

The activities of the Waste Reduction and Management (WRM) SSA are heavily focused on regulation and respond to obligations under federal legislation, such as the Fisheries Act (FA) and the Canadian Environmental Protection Act 1999 (CEPA 1999), as well as international conventions. A primary driver for much of the work of the WRM SSA is subsection 36(3) of the FA, which prohibits the deposit of deleterious substances into water frequented by fish, unless authorized by regulations. Requirements under CEPA 1999 related to the management of toxic substances and the management of hazardous and non-hazardous waste are also major drivers. While much of the WRM activity is focused on water pollution prevention, some activities (such as the management of hazardous wastes) include prevention of releases to other media. As such, the WRM SSA develops, implements and administers strategies, instruments and programs in order to manage the risks to the environment and human health from:

  • water pollution from a range of sectors (including forest products, mining and agriculture) related to the industrial production phase of materials and products to their end-of-life treatment/disposal and movement;
  • the deposit of wastewater effluent to water from wastewater systems (municipal, federal, Aboriginal and private); and
  • hazardous solid wastes, including the international and interprovincial movement and disposal of specific hazardous wastes and hazardous recyclable materials (stemming from municipal solid waste and elsewhere),[3] as per international conventions to which Canada is a signatory.

2.2 Organization, Governance and Resources

The WRM SSA of the EC PAA is one of two SSAs (the other being 3.1.2.2 – Marine Pollution) that together constitute the Waste Management sub-activity (SA). The Waste Management SA, in turn, is one of four sub-activities that contribute to the Substances and Waste Management program activity, which is aligned to the strategic outcome “Threats to Canadians and their environment from pollution are minimised.”[4]

The organizational units subsumed within the WRM SSA are located within the Environmental Stewardship Branch’s (ESB’s) Public Resources and Sectors Directorate (PRSD). The directorate, previously composed of five divisions, was merged into four divisions[5] by disbanding the Aboriginal and Public Sector Division (APSD) and realigning the APSD’s two program sections with two of the remaining PRSD divisions. The organizational units within the PRSD which perform the activities of the WRM SSA are the:

  • Mining and Processing Division (MPD);
  • Forestry, Agriculture and Aquaculture Division (FAAD);
  • Sustainable Water Management Division’s (SWMD’s) Wastewater section (previously part of APSD); and
  • Waste Reduction and Management Division (WRMD), including the APSD’s Federal House section.

The MPD is headed by an Executive Director, while each of the remaining three PRSD divisions aligned to the WRM SSA is headed by a Director. All division heads report directly to the Director General of the PRSD, who in turn reports to the Assistant Deputy Minister (ADM) of the ESB.

Two of the divisions, FAAD and MPD, are also involved with air and greenhouse gas emissions related to the forest products sector and mining and processing activities. The associated responsibilities and resources for this work are aligned with a different SSA (3.2.1.1 - Industrial Sector Emissions), however, and are therefore outside the scope of this evaluation.

The following tables present financial information in terms of budgeted (Table 1) and actual (Table 2) operations & maintenance (O&M), grants and contributions (G&Cs), salary, and full-time equivalents (FTEs) aligned to the WRM SSA for the 2007-2008 to 2010-2011 fiscal years.

Table 3 represents the applicable current and proposed regulations under the FA and CEPA 1999, which fall under the responsibility of each of the divisions of the WRM SSA.

Table 3: Regulations Relevant to Waste Reduction and Management Activities
Effective DateMining and Processing Division
2002Metal Mining Effluent RegulationsMMER (FA)
 Forestry, Agriculture and Aquaculture Division
1992Pulp and Paper Effluent RegulationsPPER(FA)
1992Pulp and Paper Mill Defoamer and Wood Chip Regulations (CEPA)
1992Pulp and Paper Mill Effluent Chlorinated Dioxins and Furans Regulations (CEPA)
 Sustainable Water Management Division
ProposedProposed Wastewater System Effluent Regulations-WSER(FA)
 Waste Reduction and Management Division – Environmental management on federal land (Federal House)
2008Storage Tank Systems for Petroleum Products and Allied Petroleum Products Regulations (CEPA)
 Waste Reduction and Management Division – Storage, disposal and use of harmful substances
2008PCB Regulations(CEPA)
1996PCB Waste Export Regulations (CEPA)
1990Federal Mobile PCB Treatment and Destruction Regulations (CEPA)
ProposedProposed Regulation for Mercury-Containing Products (Regulations Respecting Products Containing Certain Substances Listed in Schedule 1 to CEPA)
 Waste Reduction and Management Division – Transboundary movement of hazardous waste and hazardous recyclable materials
2005Export and Import of Hazardous Waste and Hazardous Recyclable Material
Regulations
-EIHWHRMR(CEPA)
2002Interprovincial Movement of Hazardous Waste Regulations (CEPA)

In addition to the regulations identified above, a number of other dated regulations and guidelines under section 36 of the FA also authorize the deposit of deleterious substances. These regulations are listed below.[14]

  • Petroleum Refinery Liquid Effluent Regulations and Guidelines
  • Chlor-Alkali Mercury Liquid Effluent Regulations
  • Meat and Poultry Products Plant Effluent Regulations and Guidelines
  • Potato Processing Plant Liquid Effluent Regulations and Guidelines

An overview of the activities of each of the four divisions is presented below.

Forestry, Agriculture and Aquaculture Division (FAAD)

FAAD leads the review and management of environmental risks related to water pollution from the forest products sector, and to a lesser extent, the agriculture and aquaculture sectors, including chemicals identified under the Chemicals Management Plan (CMP). FAAD provides subject matter expertise and leadership in the development and implementation of environmental risk management instruments for this sector, including the development of policy and regulations and the provision of advice, guidance and standards of practice in support of a nationally consistent delivery of regulations.

For the forest products sector, FAAD leads the ongoing implementation of the Pulp and Paper Effluent Regulations (PPER), under the FA, including amendments (as required). FAAD also works with the Environmental Protection Operations Directorate (EPOD) (headquarters and regions) and the Enforcement Branch on compliance promotion and enforcement activities, leading the Intradepartmental Pulp and Paper Working Group, and compliance reporting and technical analysis/guidance, including liaison with legal services and regional programs, to ensure a clear understanding of the PPER. FAAD is also responsible for the ongoing implementation of relevant CEPA 1999 regulations[15] and provides secretariat support to and engages with the Forest Products Sector Water Working Group, chaired by industry and environmental non-governmental organizations (ENGOs).

FAAD also leads the development and delivery of Environmental Effects Monitoring (EEM) programs, which are part of the regulatory requirement for the PPER and MMER. Through the National Environmental Effects Monitoring Office (NEEMO), FAAD provides leadership, monitoring, tracking and analysis to support the divisions in this SSA (excluding WRMD). NEEMO, with support from EPOD regions, reviews the information and data submitted by the regulated industries, and provides detailed scientific and technical guidance to assist the regulated sectors in meeting their obligation to conduct scientifically sound EEM studies.

FAAD’s responsibilities pertaining to the agricultural sector are focused on water quality. They include reviewing outdated FA regulations related to meat and poultry products and potato processingand other issues in the agriculture sector linked to the FA, as well as the provision of knowledge and expertise regarding the environmental impacts of agriculture sector activities, largely linked to knowledge and past involvement in the National Agri-Environmental Standards Initiative (NAESI), and ongoing work to more clearly define the division’s mandate and role with respect to the agriculture sector.

For the aquaculture sector, as part of FA work, FAAD is responsible for providing general briefings and responses to inquiries about the environmental impacts of aquaculture, and coordinating departmental responses to policy and regulatory initiatives such as the Department of Fisheries and Oceans’ (DFO’s) Pacific Aquaculture Regulations and DFO’s development of the Fish Pathogen and Pest Treatment Regulations under the FA.

FAAD also plays a broad, cross-sectoral role with respect to coordination of the Department’s responsibilities under the Fisheries Act-Pollution Prevention Provisions (FA-PPP) by serving as the focal point within the Department for the FA-PPP. A major focus of this role is working with colleagues across the Department to meet commitments in response to recommendations made in a Commissioner of the Environment and Sustainable Development (CESD) audit conducted in the spring of 2009.[16]

Mining and Processing Division (MPD)

The key waste-related regulatory function performed by the MPD is the administration of the MMER. These regulations (under the FA) are designed to protect fish and fish habitat by setting standards for metal mine effluent releases into the environment, and requiring facilities to monitor effluent at the point of discharge in order to study the effects of the effluent discharge on the environment and report on these matters to EC. In addition, the MMER includes the authority to deposit waste rock or deleterious effluent into a tailings impoundment area (TIA), as set out in Schedule 2 of the MMER. In 2008, the MMER applied to 98 mining facilities across the country.[17]

MPD activities related to the administration of the MMER include:[18]

  • Policy Development: contributing to initiatives related to streamlining the MMER and developing guidance documents.
  • Regulatory Development: providing technical advice on issues related to mining sector operations, effluents, and testing procedures.
  • Regulatory Amendment: updating the provisions within the regulations themselves, as well as responding to requests by mine owners or operators to have natural bodies of water listed as TIAs[19]; publishing proposed amendments in the Canada Gazette.[20]
  • Compliance Promotion: providing expertise and other information to the regional offices for activity 3.3 of the Department’s PAA (Compliance Promotion and Enforcement – Pollution) so that they can provide information to the regulated community, as well as compliance promotion site visits to mines, prospective mines, and operating facilities subject to the regulations or undergoing environmental assessments.[21]
  • Monitoring:supporting the work of NEEMO, receiving and analyzing reports submitted by mining and processing industry owners and operators in order to determine the effectiveness of the regulations.
  • Reporting: publishing MMER status reports and revisions to guidance documents.
  • Tracking and Support of Federal and Harmonized (Federal-Provincial) Environmental Assessments: tracking the status of mining environmental assessments across the country, documenting the roles and responsibilities of those involved in environmental assessments of mining projects, reviewing and providing comments on environmental assessments, and tracking and participating in Major Projects Management Office (MPMO) activities related to environmental assessments.

The waste reduction and management activities of MPD are performed in the national office. The Atlantic, Quebec, Ontario, Prairie and Northern, and Pacific and Yukon regional offices, which report to EPOD and the Enforcement Branch, carry out compliance promotion and enforcement activities.

Sustainable Water Management Division (SWMD)-Wastewater

SWMD’s wastewater section is responsible for the development, delivery and coordination of risk management programs and activities for the wastewater sector across Canada. All federal actions aim to protect ecosystem health, human health and fisheries resources in Canada while taking into account the actions of other jurisdictions and other federal regulators. Although the primary focus of current activity is on regulatory actions for wastewater effluents, the division is also involved in the implementation of the CEPA 1999 pollution prevention planning instrument for chlorine,[22] the wastewater aspects of the CMP and other collaborative work with Canadian Council of Ministers of the Environment (CCME) on the management of wastewater treatment sludges and biosolids.

In Canada, all levels of government share the responsibility for managing the collection, treatment and release of wastewater effluent. Most wastewater systems in Canada are owned and operated by municipalities; however, wastewater systems are also located in Aboriginal communities and the federal government also owns, operates or funds a number of wastewater systems. Effluents from wastewater systems must comply with all applicable federal legislation and associated regulations, including CEPA 1999 and the FA, as well as applicable provincial, territorial or water board legislation, regulations, permits and licences.

This shared jurisdiction for the management of wastewater has produced regulatory regimes that are inconsistent across the country, resulting in varying levels of treatment. Interested parties have consistently indicated the need for all levels of government to develop a harmonized approach to managing the sector. To address this situation, the federal government engaged with other jurisdictions to develop the CCME Canada-wide Strategy for the Management of Municipal Wastewater Effluent (CCME Strategy) endorsed in February 2009 by all jurisdictions in Canada except Quebec, Nunavut and Newfoundland and Labrador. All jurisdictions are actively engaged in its implementation. To deliver on federal commitments under the CCME Strategy, proposed Wastewater Systems Effluent Regulations (WSER) were published in the Canada Gazette, Part I, on March 20, 2010. The proposed regulations set national baseline effluent quality standards achievable through secondary treatment or its equivalent and also include monitoring and reporting requirements. The regulations would take a first step toward managing sewage overflows. The objective of the regulations is to reduce the risks to ecosystem health, fisheries resources and human health by decreasing the level of harmful substances discharged to surface waters.

Waste Reduction and Management Division (WRMD)

WRMD develops and implements pollution prevention notices and risk management strategies and measures to ensure that wastes, recyclables and substances of concern that wind up in the waste stream are managed in a manner that protects the environment and human health. The development, delivery and coordination of waste reduction and management programs and activities for Canada build upon and are consistent with the federal government’s authorities under CEPA 1999. The scope of WRMD’s work includes hazardous and non-hazardous wastes and recyclables from all sectors other than natural resource sectors (e.g., mining, forestry, agriculture) and sectors already addressed by other EC initiatives. On these subjects, WRMD provides scientific, technical and regulatory expertise, as well as policy leadership, in the development and implementation of domestic and international programs.[23] WRMD also develops and implements risk management instruments, under CEPA 1999, pertaining to selected hazardous products and products containing toxic substances, including PCBs and mercury, and administers the federal regulatory requirements with respect to the international movements of hazardous wastes and hazardous recyclables. Furthermore, WRMD provides a national and global perspective on waste issues and trends to its partners and stakeholders.

Currently, the division is responsible for administering the PCB Regulations and the Notice Requiring the Preparation and Implementation of Pollution Prevention Plans in Respect of Mercury Releases from Dental Amalgam Waste. WRMD is also in the process of amending the regulatory framework for the movement of hazardous waste, hazardous recyclable materials, and non-hazardous waste within and outside Canada, as well as developing two regulations to manage mercury-containing products. In addition, WRMD administers the Export and Import of Hazardous Waste and Hazardous Recyclable Material Regulations (EIHWHRMR) and the Interprovincial Movement of Hazardous Waste Regulations under CEPA 1999. With respect to these regulations, the division’s activities include reviewing notices and issuing permits for the movement of hazardous wastes and hazardous recyclable materials, providing technical advice on waste classification, disseminating information to stakeholders, communicating applications to international partners, tracking the movement of hazardous wastes and hazardous recyclable materials, and reporting on regulatory activities.

Federal House

Much of the direct responsibility for the regulation and provision of pollution prevention measures lies with provincial, territorial and municipal levels of government. Laws enacted by these levels of government, however, do not generally apply to “federal departments, Crown corporations, boards and agencies, federal works and undertakings, federal lands, and Aboriginal lands and persons on those lands,” which are collectively known as the Federal House.[24]

Activities performed within the Federal House Section are focused on the development of regulatory and non-regulatory initiatives designed to strengthen environmental management within the Federal House, such that the “... [e]nvironmental performance of federal house meets or exceeds federal environmental laws and is compatible with provincial, territorial and municipal standards for the jurisdiction in which they are located.”[25]

WRMD’s responsibilities with respect to the Federal House are primarily focused on:

  • Promoting/clarifying the application of CEPA 1999 Part 9 provisions; and
  • administrating EC’s Storage Tank Program, which includes the Storage Tank Systems for Petroleum Products and Allied Petroleum Products Regulations.

The Storage Tank Systems for Petroleum Products and Allied Petroleum Products Regulations are intended to reduce the risk of contamination of the environment from fuel storage tank system spills and leaks from storage tank systems on federal and Aboriginal lands across Canada.[26]

2.3 Program Logic Model

Table 4 is a visual representation of the waste reduction and management SSA, and identifies the linkages between activities and outcomes. The logic model presented was adapted from PAA element descriptions, the departmental Performance Measurement Framework for 2010-2011, Departmental Performance Report information for 2008-2009, and previous logic models completed for the FA-PPP and WRMD’s Waste Management and Reduction Program Risk Management Instruments.

This logic model was developed by the Evaluation Committee for the evaluation.

 

3.0 Evaluation Design

3.1 Purpose and Scope

The purpose of this evaluation is to assess the relevance and performance (including effectiveness, efficiency and economy) of the activities of the WRM SSA at EC. The evaluation covers the four-year time frame from 2007-2008 to 2010-2011, while also addressing significant developments which occurred in the first quarter of 2011-2012. The evaluation findings will be used to assist the ESB in the ongoing management and direction of activities related to waste reduction and management.

The WRM SSA represents 1.54% of the Department’s direct program spending, and was identified in the Department’s annual risk-based Evaluation Plan for 2010-2015 as a high-risk item, as well as an opportunity to test the feasibility of evaluating an entire PAA element in a single evaluation.[27] Additionally, a Performance Measurement and Evaluation Plan drafted by the Wastewater Section states that this evaluation “will include an assessment of the Wastewater System Effluent Regulations and will address issues related to relevance and performance.”[28] As these regulations have yet to be posted to the Canada Gazette, Part II, and therefore have not yet been enacted, evaluation evidence related to the WSER is limited to an assessment of the regulations’ relevance and the efficiency of the regulatory development process. The evaluation also assesses the relevance and performance of the transfer payment programs under this SSA.

The objectives of the evaluation were to examine and make recommendations with respect to whether:

  • the WRM SSA continues to be consistent with federal priorities and address actual needs (relevance);
  • the WRM SSA is meeting its intended outcomes (effectiveness);
  • the most appropriate and least costly means are being used to achieve program outputs (efficiency); and
  • the most appropriate and cost-effective means are being used to achieve program outcomes (economy).

3.2 Evaluation Approach and Methodology

The following section provides a detailed description of the research activities undertaken for the evaluation. Three key considerations played a role in the design of the evaluation approach and methodology used for this evaluation.

First, as noted previously, this evaluation was a pilot for a broader PAA-level evaluation. The WRM SSA cuts across four divisions addressing different regulations, stakeholders and outcomes, and as such, the evaluation methodology was designed to ensure that the data collection could support analysis at both the division and overall PAA levels.

Second, the evaluation of the WRM SSA follows several other audits and evaluations that were conducted between 2007 and 2009. These include external studies by the CESD[29] and the Auditor General of Canada,[30] as well as an implementation review of the Environmental Effects Monitoring Program,[31] and several internal evaluations that relate to different aspects of the WRM SSA.[32] This evaluation took into account the findings of these previous audits and evaluations, including relying on recent evidence as presented in these studies’ reports, to the degree it was appropriate and feasible, so as to avoid duplication and minimize respondent burden.

Finally, the evaluation took extra efforts to understand the perspective of the regulated community, through both surveys and interviews, since ultimately the success of these regulations is determined by the degree to which the regulations are understood and implemented.

With these considerations in mind, the following methods were used in the conduct of this evaluation, ensuring that the evaluation questions (see Annex 1) were addressed with multiple lines of evidence, wherever possible.[33]

Document Review

To develop a program profile and address all evaluation questions, the evaluation team reviewed existing documentation relating to waste reduction and management. Examples of documents examined include performance reports, existing legislation, regulations, policy documents, annual reports, previously conducted audits and evaluations, and risk-management tools. These documents were identified and compiled through online searches, recommendations given during interviews, and supplemental documentation provided by the Evaluation Committee.

A media scan was also conducted to address evaluation questions pertaining to relevance, effectiveness, efficiency and economy (1, 4, 6, 7, 8 and 9). The evaluation team submitted a request to the Media and Public Relations Division of the Communications Branch to identify relevant news articles related to waste reduction and management.[34] The request contained a list of English and French search terms for the various sectors and yielded 517 articles. The evaluation team also used the same search criteria on the Canadian Broadcasting Corporation’s (CBC’s) website to identify media articles, which yielded an additional 45 articles. Of the 562 articles retrieved and scanned for relevance, 217 were deemed to be relevant and used in the document review. Table 5 provides a breakdown by sector of the total articles received and those that were relevant to the evaluation.

Table 5: Breakdown of Articles Found in Media Scan
SectorTotal Articles ReceivedRelevant Articles
Agriculture118
Aquaculture91
Forestry248
Metal and Mining2911
Wastewater9138
Aboriginal Issues325101
Hazardous Waste2815
Additional CBC Articles4535
Total562217

 

Literature Review

To address evaluation questions regarding relevance, efficiency and economy (1, 8 and 9), the evaluation team conducted a review of existing research and studies on waste reduction and management. In particular, the literature review aimed to explore the appropriateness and relevance of regulatory interventions, and identify best practices and alternative program approaches for the delivery of waste reduction and management services.

To conduct the literature review, peer-reviewed articles were obtained from the University of Victoria’s journal collections. The evaluation team created a list of search terms used to find articles in journal databases, focusing on those published after 2003 and whose full texts were available online. The first forty articles obtained from each individual search were then sorted for relevancy based on the information provided in their abstracts. Approximately 42 articles were deemed to be relevant and were analyzed to address evaluation questions.

Administrative File Review

To address evaluation questions pertaining to effectiveness, efficiency and economy (4, 6, 8 and 9), the evaluation team completed a review of all G&Cs associated with waste reduction and management. In total, this SSA has eight grants and contributions, amounting to $211,000, the majority of which fund international projects such as Canada’s obligations under the Basel Convention and the Organisation for Economic Cooperation and Development (OECD) Working Group on Waste Prevention and Recycling. Contribution agreements, project activity reports, financial files and performance reports for each G&C were compiled and examined to establish how government funds were spent and whether the funded activities had clearly achieved their expected outcomes and outputs.

An administrative file review of EPOD compliance promotion activities for regulated sectors was also carried out in order to gain a better understanding of the types of activities that are undertaken to raise regulatees' awareness and understanding of regulatory obligations, with the objective of improving overall compliance. The review examined documents related to priority planning and compliance activities, as well as compliance results for WRM-related regulations as reported in NEMISIS.[35]

Key Informant Interviews

Key informant interviews contributed to all evaluation questions. The evaluation committee provided the evaluators with a list of approximately 175 potential interviewees, each of whom either represented or interacted with the four divisions included in this evaluation. A targeted sample was selected from the universe of potential interviewees to ensure all divisions and stakeholder groups were represented in the final respondent sample. Launch memos were sent to 96 interviewees. In total, the EC evaluation team, with support from Beechwood Consulting & Research, conducted 62 semi-structured interviews.

Interview guides were developed and utilized for each respondent group: program staff, government stakeholders, and external stakeholders. Recognizing that key informants were generally only able to respond to questions relevant to the work of one of the four divisions, as opposed to the SSA as a whole, or, in some cases, were familiar with the work but not the organization’s name, an annex was attached to the interview guide to provide additional details on the four divisions and their work. Interviewers also ensured that the relevant division with which the interviewee had interacted was clearly understood and documented for each interview. This information was important to support the divisional analysis of the interview data. Table 6 provides a breakdown of the number of interviews completed for each stakeholders group.

Table 6: Key Informant Interviewees Summary
 Stakeholder GroupNumber of Interviews Conducted
 Program Staff
13
 

Government Stakeholders

  • Other Federal Contacts (9 EC[36] & 8 other government departments [OGDs])
  • Provincial/Territorial/Municipal

 

 

17
9

 

External Stakeholders

  • Industry
  • ENGO/Academic
  • Aboriginal
  • International

 

7
6
6
4

Total 
62

 

Online Survey

An online survey was conducted with representatives from industry, ENGOs, municipalities, and provincial, territorial and federal governments, in order to contribute to evaluation questions for performance (4, 6 and 7). A list of 511 representatives was supplied by the Evaluation Committee. An email invitation with an embedded hyperlink to the questionnaire was distributed to all 511 potential respondents identified and was accessible for 12 days. A total of 150 individuals responded, representing a response rate of 31%; however, only 94 of the 150 respondents self-identified as having direct involvement with WRM’s activities. The majority of respondents were industry and municipal representatives. Table 7 provides a breakdown of respondents by sector. Table 7 lists the sectors surveyed and the number of respondents per sector. Respondents include individuals from the following areas: metal mining, pulp and paper, environmental management on federal land, wastewater, storage recovery, treatment, disposal and/or use of harmful substances and waste, transboundary movement of hazardous waste and/or hazardous recyclable material, and individuals with no involvement in any of these areas.

Table 7: Survey Respondent Summary
SectorNumber of Respondents
Metal Mining4
Pulp and Paper18
Environmental management on federal land0
Wastewater46
Storage recovery, treatment, disposal and/or use of harmful substances and waste10
Transboundary movement of hazardous waste and/or hazardous recyclable material13
No involvement in any of these areas3
Total94

The survey was programmed using FluidSurveys software and was available in English and French. The questionnaire contained a skip logic that enabled respondents to answer only questions that directly related to their sector and expertise.

3.3 Limitations

While WRM activities are not new to the Department, the WRM SSA is a new entity in the 2010-2011 PAA. As such, the SSA did not yet have a fully articulated logic model or performance measurement strategy. In order to address this, the evaluation team developed a logic model and associated outcomes for the purpose of the evaluation. While these may not represent the final indicators selected to measure performance, the evaluation team made efforts to ensure that the newly developed logic model was an effective tool for assessing performance for the SSA. This was done by basing it on existing performance documents, including logic models which had been previously developed for specific components of the SSA, and developing and validating the logic model in consultation with the Evaluation Committee. This new logic model was then used as a starting point to develop reasonable indicators to measure the SSA’s performance.

Another limitation relates to the fact that the WRM SSA is delivered by a directorate composed of four divisions which focus on different sectors and regulations, employ different terminology, and measure different outcomes. As this is an evaluation of the SSA, as opposed to an evaluation of the four separate divisions, the evaluation attempts to address issues and draw conclusions and recommendations for the SSA as a whole. For the most part, however, key informants and survey respondents were only able to respond to questions within the context of the work of one of these divisions, and did not bring a perspective on the overall performance of the SSA. This was also the case for most of the documents reviewed for the evaluation. In order to address this limitation, the evaluation methodology was developed to ensure representation across each of the divisions of the SSA, and data were analyzed first to understand the division‑level findings, and secondly to look for findings which applied more broadly at the SSA level. The report attempts to clearly identify when findings pertain to the SSA as a whole, and when there is a finding that is limited to one or more of the divisions.

Thirdly, while the survey was sent to a large sample size (over 500) and the response rate was consistent with expected response rates for online surveys (31%), the sample size was too small to provide statistical validity at the regulation level, and, additionally, response rates varied by sub-segment. For example, there was a proportionally higher response from regulatees for the proposed wastewater regulations. This may be due to the size of the wastewater sector and both the timing and high level of stakeholder engagement, in particular since the regulations were published in the Canada Gazette, Part I, in March 2010. Given the very small response rates for some regulations, and given that full population sizes for each sub-segment were not known, results were not weighted. The findings from this line of evidence, therefore, should be viewed as illustrative and interpreted with caution. To compensate for this, the survey data were used in a limited fashion, and the evaluation also relied on additional sources of evidence.

Finally, the manner in which the Department’s financial information is coded prevented a full understanding of the contribution to this SSA from support functions, and a complete analysis of planned versus actual expenditures, thereby limiting the analysis of economy and efficiency within the evaluation.

3.4 Lessons Learned from a PAA Level Evaluation

In response to the 2009 Treasury Board Policy on Evaluation requirement to evaluate 100% of departmental direct program spending every five years, Environment Canada’s Evaluation Division employed a new approach to planning and conducting the Waste Reduction and Management evaluation. Unlike the traditional approach to evaluation, whereby individual programs were selected for evaluation according to risk criteria, the new approach defines the universe of evaluable entities within the department according to the department’s PAA. The PAA is “an inventory of all the program activities undertaken by a department, depicted in their logical relationship to each other and to the strategic outcome(s) to which they contribute.”[37] Thus, the use of the PAA as an instrument to define the evaluation universe ensures that all direct program spending can be accounted for in the context of the department’s five-year evaluation planning, and that evaluation coverage of direct program spending is maximized within available resources. This new approach involves the evaluation of the lowest identifiable PAA entities, called sub-activities (SAs) or sub-sub-activities (SSAs).

The evaluation of the Waste Reduction and Management SSA was among the first evaluations conducted by the Evaluation Division at Environment Canada using this approach. As such, upon completion of this evaluation, an exercise was undertaken to reflect upon the experience, and to identify lessons learned which can be applied to similar evaluations in the future. The following represents the lessons learned from this evaluation both from the evaluators’ and evaluatees’ perspectives. The first lesson raises questions about the value of always conducting evaluations of PAA entities, while the other two provide lessons for subsequent evaluations of SAs or SSAs.

SA or SSA evaluations may not always be appropriate in circumstances where there is considerable variability among the program elements within the PAA entity.

The level of complexity of SA or SSA evaluations seems clearly linked to the homogeneity of the programs within the entity. This evaluation included a mix of programs supporting regulations related to the Fisheries Act, as well as regulations based on CEPA 1999, two pieces of legislation which take different regulatory approaches. Additionally, this SSA included both regulations which had been in existence for a considerable time, as well as others which are still in the development phase. In cases such as this, where the work of the PAA entity is particularly diverse, consideration should be given to whether the principles of value and efficiency would be better served through separate evaluations for dissimilar SA or SSA components.

Although there are benefits to investing time in the development of a comprehensive logic model for an SA or SSA, there may be instances when employing multiple logic models and/or relying on existing logic models for the individual program elements within a PAA entity may be more appropriate.

A comprehensive SA or SSA-level logic model, which presents the combined activities, outputs and intended outcomes for all program areas within the PAA entity, can be a useful tool. Logic models developed at this level provide a framework for reporting that is consistent with departmental reporting requirements (e.g., Departmental Performance Report, Strategic Reviews), are better aligned to evaluation planning, and present an overview of how the various components of the entity fit together and that many program managers find useful.

In some instances, however, the use of a single PAA-level logic model may not be appropriate. The Waste Reduction and Management SSA was first introduced to the department’s PAA in 2010-11, when planning for the evaluation began. As such, the evaluators’ first task was to develop a logic model for the SSA in order to clarify the theoretical framework against which the SSA would be evaluated. While many SAs or SSAs are comprised of only one or a small set of relatively homogeneous programs, many others are not. For example, the Waste Reduction and Management SSA is comprised of four divisions, each of which focus on different outcomes and employs different activities to achieve these. As a result of the newness and heterogeneity of the SSA, significant resources were invested on the part of both the Evaluation Division and the four SSA divisions in trying to develop a conceptual basis for linking the activities, outputs and outcomes of the different divisions in order to show how they contribute to shared SSA-level outcomes. Ultimately, it was not possible within available timelines and resources to fully reconcile and integrate the work of all four divisions. Rather, draft logic models for two subsets of the SSA (water quality and waste) were used as the basis for the evaluation.

A risk-based approach to evaluating a PAA entity would allow evaluators to balance evaluation coverage requirements (as per the Policy on Evaluation) with the information needs of managers by focussing evaluation resources on areas of greatest risk.

The Waste Reduction and Management SSA is delivered by a Directorate comprised of four divisions, which focus on different sectors and regulations, employ different terminology, and measure different outcomes. The evaluation was designed to provide the same depth of analysis on the work of all four divisions and draw conclusions and recommendations at a higher, more strategic level. Where evidence suggested that there were issues of concern, however, this approach did not yield sufficient information to clearly identify and support conclusions related to the risk. Rather than evaluating all activities within a given SA or SSA in the same manner, an approach which affords more in-depth analysis of higher risk areas and shallower analysis of lower risk areas (e.g., areas of activity that have been evaluated previously or are of lower materiality), likely would yield more actionable findings and conclusions.

In response to lessons learned from the Waste Reduction and Management SSA evaluation, the Evaluation Division has revised its approach to conducting pre-evaluation checks in order to better respond to the particular challenges of PAA-level evaluations. First initiated in 2010-11, pre-evaluation checks are undertaken one to two years prior to a planned evaluation in order to assess the evaluability of a PAA element by exploring issues such as the availability of performance data, as well as the existence of a clearly defined program theory (e.g., logic model) and governance model. In this way, the PECs identify issues that, if addressed by the evaluatees prior to the evaluation, can greatly reduce program and evaluation burden and yield better forecasts of resources required to support the evaluation. The process for future PECs has been refined to focus in part on how best to scope the PAA-level evaluations to ensure that evaluation resources are better aligned to areas of greater risk. In addition, efforts will be made to better engage ADMs and other senior managers during the evaluation planning phase to ensure evaluations of programs for which they are accountable are adequately scoped and address their information needs.

4.0 Findings

The findings of this evaluation are presented below by evaluation issue (relevance and performance) and by the related evaluation questions. The findings at the overall issue level are presented first, followed by the findings for each evaluation question.

A rating is also provided for each evaluation question based on a judgement of the evaluation findings. The rating symbols and their significance are outlined below in Table 8. A summary of ratings for the evaluation issues and questions is presented in Annex 2.

Table 8 displays the scale used to rate the degree to which relevance and performance has been achieved by the program. These ratings include: Achieved; Progress Made, Attention Needed; Little Progress, Priority for Attention; and Not Applicable. The tilde (~) symbol denotes that although there is compelling subjective evidence in support of the rating, a complete assessment cannot be done due to a lack of performance data.

Table 8: Rating Symbols and Significance
SymbolSignificance
AchievedThe intended outcomes or goals have been achieved or met.
Progress Made; Attention NeededConsiderable progress has been made to meet the intended outcomes or goals, but attention is still needed.
Little Progress; Priority for AttentionLittle progress has been made to meet the intended outcomes or goals, and attention is needed on a priority basis.
N/AA rating is not applicable.
~Addition of this symbol before a rating means that, although there is compelling subjective evidence in support of the rating, a complete assessment cannot be done due to a lack of performance data.

 

4.1 Relevance

Evaluation Issue: Relevance

Does the program remain consistent with and contribute to the federal government priorities and address actual needs?

Overall Findings:

Waste reduction and management activities are consistent with and contribute to federal government priorities, and clearly address an actual need to manage risks to the environment and human health through the provision of key regulations and non-regulatory instruments in the area of water quality and waste.

Continued Need

Evaluation Issue: RelevanceIndicator(s)MethodsRating
1. Is there a continued need for theprogram?
  • Demonstration of societal/ environmental need
  • Presence/absence of other programs that complement or duplicate the objectives of the program
  • Reach and activities are connected to societal/environmental needs
  • Document Review
  • Key Informant Interviews
  • Literature Review
Achieved

There is a continued need for WRM activities in order to manage risks to the environment and human health from hazardous solid waste, toxic substances, water pollution from a range of sectors, and the deposit of wastewater effluent to water from wastewater systems. Although the provinces have parallel regulations in place to govern many of the same issues, the federal government’s role in providing national coordination, as well as the fact that the federal regulations apply to different jurisdictions, suggests that the risk management activities of the WRM SSA are complementary, as opposed to duplicative.

The WRM SSA plays an important role in assessing the risks of toxic substances, including the hazards they present and routes of exposure, and in managing those risks to prevent risks to human health and the environment. While much of the WRM activity is focused on water pollution prevention, some activities such as the management of hazardous wastes include prevention of releases to other media. More specifically, the WRM SSA addresses the need to manage the risks to the environment and human health from:

  • water pollution from a range of sectors, including forest products, mining and agriculture, related to the industrial production phase of materials and products to their end-of-life treatment/disposal and movement;
  • the deposit of wastewater effluent to water from wastewater systems (municipal, federal, Aboriginal and private);
  • hazardous solid wastes, including the international and interprovincial movement and disposal of specific hazardous wastes and hazardous recyclable materials (stemming from municipal solid waste and elsewhere);[38] and
  • toxic substances, including risk management of substances under the CMP for the WRM SSA sectors (e.g., Bisphenol A, mercury).

Available evidence suggests a clear need to address these risks:

  • The pulp and paper industry rates second in effluent output to the Canadian environment, releasing significant discharge to aquatic environments.[39]
  • Aquaculture accounts for 25% of Canada’s fish and seafood production, and poses several risks to the environment related to water quality, fish habitat and health.[40]
  • Metal mining facilities (~ 104 facilities) generate tailings and effluents, and water management constitutes a primary environmental concern at mines.[41]
  • Wastewater effluents from systems located in municipalities and communities, including those on federal and Aboriginal land, are the largest source of pollution to surface water in Canada. The effects of untreated and under-treated wastewater include negative impacts on fish and wildlife populations, recreational water use, fish/shellfish harvesting and consumption, and drinking water.[42]
  • A 2006 CCME report identified 69 substances of concern in municipal wastewater effluent (exceeded safe concentrations).[43]
  • Hazardous materials generated by a range of sources (industrial, manufacturing, medical) must be recycled/disposed of at authorized facilities.[44]
  • Mercury is currently a pollutant of global concern (released through mining, smelting, coal burning and product use/disposal).[45]
  • Approximately 6 million tonnes of hazardous waste are produced in Canada each year. Further, approximately 417 000 tonnes of hazardous waste are imported each year (~ 55% destined for recycling) and 320 000 tonnes are exported each year (~ 65% destined for recycling).[46]
  • PCBs are of concern due to their environmental and health effects and their persistence in the environment and living tissue.[47]
  • Contamination of soil and groundwater caused by leaks and spills of petroleum products and allied petroleum products from storage tank systems is the highest single source of contaminated sites on federal and Aboriginal lands in Canada.[48] The storage tank regulations stemmed from the need identified during the birth of the Federal Contaminated Sites Action Plan, which found that a majority of contaminated properties were a direct result of improper storage tank management.[49]

Regarding the presence of other programs that complement or duplicate the objectives of the WRM SSA, it is clear that the provinces also play an important role in regulating effluents and waste. In many cases, there are parallel provincial regulations in place which govern the same issues, which some interviewees believed was an overlap with the provinces. However, the work of the WRM SSA also includes a coordinating role among the provinces, and some of the federal regulations serve to address the regulatory gap on federal and First Nations’ lands. Findings on this issue included:

  • Regarding FAAD, some interviewees believed there was overlap/duplication between the FA-PPP and provincial regulations governing pulp and paper mills. The intent is for the federal government to have regulations and standards to set national allowable limits; however, provinces may decide to put more rigorous limits in place and this may be perceived as duplication.
  • With respect to MPD, the provinces play a role in regulating mining effluent. For example, 21 provincial acts and 12 regulations govern the mining industry in Ontario alone.[50] However, there was little agreement among interviewees on whether this is overlapping or complementary.
  • In the area of SWMD’s responsibilities, different levels of government manage risks associated with wastewater by setting limits on substances contained in, and standards for the treatment of, effluent. The CCME developed a Canada-wide Strategy for the Management of Municipal Wastewater Effluent to address the need for a coordinated approach in this area.[51] Interviewees indicated that the work of CCME was effective in increasing harmonization across Canada.
  • The proposed wastewater regulations will play an important role in addressing regulatory environmental gaps on reserves, as previously there were no federal wastewater regulations that applied on reserves.[52]
  • While it was identified by interviewees that activities under WRMD are duplicated within some provinces, EC, while working with other jurisdictions, is taking the lead to develop a national instrument for end-of–life management of mercury‑containing lamps, which are intended to reduce environmental and health risks to Canadians.

Alignment to Government Priorities

Evaluation Issue: RelevanceIndicator(s)MethodsRating
2. Is the program aligned to federal governmentpriorities?
  • Program objectives correspond to recent/current federal government priorities
  • Program objectives are aligned with current departmental strategic outcomes
  • Views on the alignment of program objectives with recent/current federal and departmental priorities
  • Document Review
  • Key Informant Interviews
Achieved

The Department’s activities in waste reduction and management are aligned with federal government priorities and departmental strategic outcomes. More specifically, activities of WRM contribute to the general health and safety of all Canadians as well as environmental health and ecosystem resilience in Canada, and support environmentally sustainable economic development. Further, the activities of the WRM SSA provide support to the following recently articulated federal government priorities:

  • water quality, and a reduction of risks associated with wastewater effluents;
  • Northern strategy: highlighting pollution prevention and waste management activities in the North; and
  • commitment to clean drinking water and effective wastewater management on First Nations’ reserves.

The document review and interviews with program staff from all four divisions affirm that the WRM activities reflect government priorities and align with the Department’s strategic outcomes. In terms of federal priorities:

  • The 2010 Budget committed to an investment in the forest sector through the Next Generation Renewable Power Initiative.[53]
  • The 2010 Speech from the Throne confirmed that access to drinking water and effective wastewater treatment on-reserve was a national priority, and promised the introduction of legislative measures to further this goal.
  • The Federal Sustainable Development Strategy includes targets to reduce risks associated with wastewater effluent by 2020.[54]
  • The 2007 Speech from the Throne featured an integrated northern strategy that included measures to improve elements of infrastructure, including water treatment.

The activities of the four divisions of PRSD have a primary focus on the reduction of pollution in water and the management of toxic substances. Consequently, they address the departmental strategic outcome: risks to Canadians, their health and their environment posed by toxic and other harmful substances are reduced; and they contribute to EC’s broad strategic outcome: Canadians and their environment are protected from the effects of pollution and waste. Additionally, the water quality work also contributes to the Department’s strategic outcome: Canada’s natural capital is restored, conserved and enhanced.[55]

In addition, key informant interviewees identified several other ways in which the WRM SSA activities contribute to federal priorities:

  • MPD activities support the current federal focus on economic development, to be undertaken in an environmentally sustainable manner.
  • SWMD’s wastewater group, along with Aboriginal Affairs and Northern Development Canada (AANDC) and Industry Canada share the support function for the federal priorities of the CCME as well as Canada’s related international obligations.
  • WRMD activities align with departmental priorities, as well as to national commitments (CCME, CMP, Northern Development Strategies) and international commitments (Basel Convention, OECD).

Consistency with Roles and Responsibilities

Evaluation Issue: RelevanceIndicator(s)MethodsRating
3. Is the program consistent with federal and departmentalroles andresponsibilities?
  • Program mandate aligned with federal government jurisdiction
  • Evidence on the appropriateness of federal involvement
  • Document Review
  • Key Informant Interviews
Achieved

The program is consistent with federal roles and responsibilities. There is a clear, legislated mandate for federal government involvement in the provision of waste reduction and management services, which has been established by regulations and guidelinesunder CEPA 1999 and the FA, as well as within the responsibilities of the Canada Water Act and international obligations under the Basel Convention related to the import and export of hazardous waste.

The federal government’s involvement in the provision of WRM services is consistent with the mandate which has been established by regulations and guidelines under CEPA 1999 and the FA as well as within the responsibilities of the Canada Water Act. DFO is accountable for the FA; however, EC has responsibility for subsection 36(3) of the FA regarding the disposal of deleterious substances in waters frequented by fish, in addition to lesser responsibilities for several other FA subsections.[56]

Additionally, WRM activities are necessary to meet Canada’s international obligations under the United Nations Environment Programme’s Basel Convention on the Import and Export of Hazardous Waste, specific OECD Council Decisions, and the Canada-United States Agreement Concerning the Transboundary Movement of Hazardous Wastes.[57]

4.2 Performance

Evaluation Issue: Performance

Has the program achieved its intended outcomes? Are the most appropriate, efficient and economic means being used to achieve outcomes?

Overall Findings:

The WRM SSA has made considerable progress toward its immediate and intermediate outcomes, without any significant unintended outcomes. With a few exceptions, compliance results demonstrate that compliance is high for the regulations which fall within the responsibility of the WRM SSA. Supported by the efforts of compliance promotion, most regulatees are aware of and understand the requirements of the regulations in this area. A significant amount of performance information is collected, although there is a need to continue efforts to replace manual systems with online capability and more could be done to understand the overall performance story for the SSA. Although the divisions are managing with their current resources by prioritizing their activities, resource shortages have meant that some priority items are not being fully addressed, and are having an impact on one division, WRMD, in particular with respect to delivery of its outcomes.

The WRM SSA has many processes in place which support the efficient delivery of its outputs, including governance and priority-setting processes, which are generally viewed as appropriate, and extensive collaboration and consultation, which actively engages stakeholders. Opportunities to improve efficiency were identified with respect to the clarity of roles between EC and DFO as well as with the provinces. The regulatory approach for risk management was viewed to be an effective approach and no viable alternative delivery models were identified.

Limitations with financial information did not support an analysis of the full cost of delivering the SSA’s risk management instruments or a complete analysis of the SSA’s budget versus expenditures. Furthermore, the majority of interviewees were of the view that good value was being obtained through EC’s WRM activities.

Achievement of Outcomes

Evaluation Issue: PerformanceIndicator(s)MethodsRating
4. To what extent have intended outcomes been achieved as a result of the program?
  • Evidence of / views on achievement of intended outputs and outcome achievement
  • Evidence of / views on factors outside the direct purview of the program that may influence / have influenced the achievement of intended outcomes
  • Views on the extent to which intended outcomes have been achieved as a result of the program
  • Document Review
  • Key Informant Interviews
  • Online survey
  • File Review
Progress Made; Attention Needed

WRM has made progress toward the achievement of its immediate outcomes. In particular, progress has been made on outcomes relating to monitoring and reporting, awareness and understanding,and decreased infractions. Some issues exist with specific regulations for each of these outcomes, many of which are related to the regulations managed by WRMD. WRM has also made significantprogress with respect to its intermediate and final outcomes.

Immediate Outcomes

The following section provides the findings related to each of the six immediate outcomes identified for WRM.

a) Monitoring and reporting requirements for the management of harmful substances are met.

There is general agreement among stakeholders that EC’s WRM activities have made progress in meeting monitoring and reporting requirements, as regulatees are reporting on a regular basis as required by the regulations.

Under the FA, data is collected directly from regulatees for both the PPER and MMER by the EEM Program, in order to comply with the FA regulations. The EEM Program prepares a report every 2-3 years (by cycle), which includes an assessment by regulation. The MMER also expressly identify a requirement for EC to prepare an annual report, which results in the Summary Review of Performance of Metal Mines, providing detailed information on mine effluent quality data and includes compliance data on both an individual and summary basis, although the PPER do not have a similar requirement.

Performance data are also collected directly from regulatees as a requirement of the storage tank regulations (Federal Identification Registry of Storage Tank Systems’ database). These regulations are still in the implementation phase and it is expected that a report will be developed every 2-4 years (by cycle) and will include an assessment of the regulations.

A tracking system entitled Canadian Notice and Manifest Tracking System (CNMTS) is used to meet the more stringent controls introduced in 2005 by the new EIHWHRMR. Data are also collected from regulatees in regard to the PCB Regulations through a PCB database. The first report based on 2009 data should be available on the PCB website in 2011.

Survey respondents were also asked about the clarity of monitoring and reporting requirements for the regulations with which they had experience. Recognizing that sample sizes at the individual regulation level are very small, the survey revealed generally high ratings on the clarity of monitoring and reporting requirements, with the exceptions of the WSER,[58]EIHWHRMR and the Interprovincial Movement of Hazardous Waste Regulations.

Some key informant external stakeholders in FAAD, SWMD and WRMD expressed some disappointment in the area of monitoring and reporting, however, noting that there is a reduction in the quality and quantity of data available from EC. In the case of FAAD, there is a strong view among industry stakeholders that the toxicity[59] testing approach needs to be revised in order to address various concerns, including the possibility of false positives and the amount of water required to perform the test. FAAD is currently working with the sector to assess the validity of these claims and review potential options.

b) Increased awareness and understanding of the requirements of the Fisheries Act-Pollution Prevention Provisions and/or the Canadian Environmental Protection Act, 1999

Key informants generally agree that there are high levels of compliance promotion and engagement with industry on FA and CEPA regulations and that, as a result, awareness has been increased. This is supported by an analysis of Compliance Promotion and Analysis Division’s activities by regulation, which found a wide range of activities are undertaken by the Compliance Promotion and Analysis Division, including workshops, site visits, meetings, presentations, mail-outs, and the provision of advice. However, it should be noted that compliance promotion activities occur at the regional level and are not consistent across regions, as the population and density of regulatees differs by region. Program staff report that EC undertakes approximately two years of active compliance promotion activity as part of the implementation of each regulation in order to increase awareness. Government stakeholders also acknowledge high levels of engagement. In particular, several note that there used to be more communication from and interaction with EC and that a newsletter that used to be published by ECis no longer available. External stakeholders also feel that industry is generally well-informed, but external stakeholders who work with MPD feel that smaller firms are generally less informed than larger ones.

For the most part, survey respondents reported that the regulations/regulatory instruments with which they had experience are well communicated and clearly understood. Similar to the ratings on the clarity of monitoring requirements, the exceptions were the WSER (which are still under development) and WRMD’s regulations regarding hazardous waste (the EIHWHRMR and the Interprovincial Movement of Hazardous Waste Regulations), which received lower ratings on this issue.[60]

c) Effective inter/intra-governmental agreements are in place to streamline and coordinate legislative and regulatory activity.

The majority of survey respondents reported[61] that “responsibility for legislative and regulatory activity across different jurisdictions is clear” (74% rated “somewhat” or “to a large extent”); while only 57% reported “legislative and regulatory activity is streamlined across different jurisdictions.” This was identified as a clear area for improvement among respondents, and was further emphasized by survey responses regarding areas of improvement, where the issue of harmonization with provinces and municipalities was raised by numerous respondents, identifying, for example, that “duplication of existing provincial legislation is resulting in increased administration, through the need to respond to both federal and provincial agencies.”

EC has agreements with some provinces with respect to administration of the FA-PPP and PPER (Alberta, Saskatchewan and Quebec). These agreements are administered by EC’s Enforcement Branch.

Additionally, the shared responsibilities for the administrative and enforcement responsibilities of the FA-PPP between EC and DFO are currently outlined in a 1985 MOU between EC and DFO. Consistent with previous findings from the CESD Spring 2009 audit, which identified a need to clarify the relationship between DFO and EC for the FA-PPP, there was general agreement among interviewees from both EC and DFO that there was a need to revise this agreement for added clarity.

Although all interviewees believe that progress has been made with respect to inter/intra-governmental agreements and the coordination of legislative/regulatory activities, program staff tended to rate success more highly than other groups. External stakeholders representing industry report that the pulp and paper industry is frequently subject to double reporting (i.e., reporting to both provincial and federal governments). Program staff recognized this issue and note, for example, that there is further work required for the PPER to develop intergovernmental agreements with the provinces.

The work which took place with the CCME toward developing a Canada-wide Strategy for the Management of Municipal Wastewater Effluent[62] was identified by interviewees as being an effective means of providing a coordinated approach and increasing harmonization across Canada for these issues.

d) Decreased infractions with FA-PPP and/or CEPA 1999 requirements

Compliance results are high for nearly all regulations. Compliance results for all regulations, as reported by the Enforcement Branch in Table 9, range from 81-100% between 2007-2008 and 2010-2011, based on a targeted sample of regulatees. Compliance results are also high for the MMER[63] between 2003-2009 and the PPER[64] in 2008, based on self-reported data of the entire regulated community through the Regulatory Information Submissions System (RISS), with results ranging from 93-100%. Compliance results for the storage tank regulations were also high (~ 85%) as identitifed in 2010 through the Federal Identification Registry for Storage Tank Systems’ (FIRSTS’) database.[65]

This is further supported by interviewees who agree that EC’s WRM activities have made a contribution to decreasing infractions with FA-PPP/CEPA 1999 requirements. Regulatees strive to avoid infractions and no interviewee noted a large number of infractions. The early compliance promotion that occurs is seen as playing an important role in avoiding infractions through increased awareness and understanding.

Survey respondents also provided high ratings[66] in terms of progress on this outcome, with ninety-four percent (94%) of survey respondents reporting that compliance with regulatory and non-regulatory requirements is increasing “somewhat” (43%) or “to a large extent” (51%).

Targeted efforts on the part of WRMD regarding the storage tank regulations are a positive example of how WRM activities have successfully contributed to improved compliance results (i.e., decreased infractions). Discussions among the Tank Network in advance of the first regulatory deadline had highlighted low compliance results among First Nations regulatees. In response, in 2010 the Storage Tank Program, with the assistance of AANDC, undertook extensive targeted compliance promotions among First Nations communities, which resulted in increasing compliance results for these regulations among First Nations, from approximately 3%[67] in January 2010 to approximately 85%[68] in December 2010.[69]

While generally the compliance results from different sources were complementary, data for the PCB Regulations[70] were an example where diverging results made it difficult to understand the degree to which progress was made on this outcome. This sector is deemed to be at high risk for non-compliance due to two significant factors: limited / lack of funding among regulatees to implement requirements, and short timelines proposed to meet regulatory requirements. The Compliance Promotion and Analysis Division and the PCB program currently estimate that compliance regarding reporting requirements, end-of-use and the end-of-storage deadline is currently lower than 50% in most regions.[71] This is based on data from known regulatees who received compliance promotion information and who stated that they were not previously aware of the new requirements under the amended PCB Regulations. This finding is echoed by the fact that 50% of inspected regulatees also claimed a lack of awareness of the new amendments.[72] Enforcement data from NEMISIS for 2010-2011, however, shows compliance rates of 91.9% for the PCB Regulations based on 537 inspections (see Table 9). This was based on targeted inspections of a segment of known regulatees such as owners of large food and feed processing facilities who are more likely to have PCB equipment/transformers in their operations, as well as pre-schools and schools, senior care facilities, hospitals and water plants where the potential consequence of contamination would be of greater severity. This data would seem to indicate that the targeted high-risk segment had high compliance results. This is a positive result, but illustrates the complexity of understanding compliance results, and the degree to which this outcome has been met.

As evidenced above, obtaining a clear picture of compliance results for the WRM regulations is often difficult. While a variety of compliance information is available for the WRM regulations, true compliance rates for specific regulations are typically not possible because the size of the regulated community is not known.[73] Furthermore, while compliance results are available from a variety of sources, these tend not to reference the subset of regulatees that are represented by the results. These include data from the Enforcement Branch, the Compliance Promotion and Analysis Division and a number of databases (see evaluation question 5 for more details).

Table 9 lists the compliance results (number of inspections and percent compliant) for WRM regulations from 2007-2008 to 2010-2011. Regulations include the following: Metal Mining Effluent Regulations, Storage Tank Systems for Petroleum and Allied Petroleum Products Regulations, Pulp and Paper Effluent Regulations, Pulp and Paper Mill Defoamer and Wood Chip Regulations, Pulp and Paper Mill Effluent Chlorinated Dioxins and Furans Regulations, Pollution Prevention Provisions of Fisheries Act, PCB Regulations, Storage of PCB Material Regulations, Waste Bundle: Export and Import of Hazardous Waste and Hazardous Recyclable Material Regulations and Waste Bundle: Interprovincial Movement of Hazardous Waste.

Table 9: Compliance Results for Waste Reduction and Management Regulations between 2007-2008 and 2010-2011 Reported by the Enforcement Branch[74]
RegulationInspections2007-20082008-20092009-20102010-2011
Source: National Emergencies and Enforcement Information System and Intelligence System (NEMISIS) tracking system
Metal Mining Effluent Regulations (Fisheries Act)Total No.691629482572
% Compliant81.91%90.78%89.63%91.08%
Storage Tank Systems for Petroleum Products and Allied Petroleum Products Regulations (CEPA)Total No.n/a1559385
% Compliantn/a100%91.53%87.01%
Pulp and Paper Effluent Regulations (Fisheries Act)Total No.1530133112301135
% Compliant91.83%93.39%93.58%95.33%
Pulp and Paper Mill Defoamer and Wood Chip Regulations (CEPA)Total No.62483139
% Compliant100%100%96.77%97.44%
Pulp and Paper Mill Effluent Chlorinated Dioxins and Furans Regulations (CEPA)Total No.129977676
% Compliant98.45%100%100%100%
Pollution Prevention Provisions of Fisheries ActTotal No.1433122516291868
% Compliant81.93%82.69%87.78%90.20%
PCB Regulations (CEPA)Total No.4113187537[75]
% Compliant100%90.27%90.37%91.90%
Storage of PCB Material Regulations (CEPA)Total No.244102n/a10
% Compliant96.72%95.10%n/a100%
Waste bundle: Export and Import of Hazardous Waste and Hazardous Recyclable Material Regulations (CEPA)Total No.753690597279
% Compliant87.36%95.73%92.52%94.50%
Waste bundle: Interprovincial Movement of Hazardous Waste Regulations (CEPA)Total No.69162651
% Compliant100%100%96.15%98.04%

e) Improved handling, disposal and recycling practices for end-of-life products and substances moving across Canadian or provincial/territorial boundaries[76]

Although interviewees believe that positive progress has been made on this outcome, less progress has been made relative to other short-term outcomes. The main issue identified was that this outcome seems more closely aligned to provincial and municipal areas of responsibility. Nevertheless, EC is seen as having an important role to play with respect to coordination across jurisdictions and the establishment of national standards.

A majority (85%) of survey respondents[77] agree that “practices for handling, disposal and recycling of end-of-life products and substances are improving” somewhat (51%) or to a large extent (34%). Similarly, four in five (82%) believe that “practices for the movement of harmful substances across Canadian or provincial/territorial boundaries are improving” either somewhat (54%) or to a large extent (28%). It should be noted that recycling rates of wastes exported are much higher than the rates of wastes that are imported into Canada.[78]

f) Canada’s international commitments with respect to harmful substances and waste are met[79]

Canada’s international commitments related to waste fall under the responsibility of WRMD and include commitments related to the transboundary movement of hazardous waste and the control of PCBs, as follows:

Transboundary Movement of Hazardous Waste
  • Basel Convention on the Control of Transboundary Movement of Hazardous Wastes and Their Disposal
  • Agreement between the Government of Canada and the Government of the United States concerning the Transboundary Movement of Hazardous Waste
  • OECD Decision Concerning the Control of Transboundary Movements of Wastes Destined for Recovery Operations
Control of PCBs
  • OECD Decision on the Protection of the Environment by Control of PCBs

Regarding the transboundary movement of hazardous waste, both the Canada-U.S. Agreement and the OECD Decision are based on the same principles as the Basel Convention and, as such, all three agreements have the key objective of minimizing the generation of hazardous wastes, ensuring they are disposed in an environmentally sound manner and as close to the source of generation as possible and reducing the international movement of hazardous waste. Compliance with the commitments of the three international agreements requires:

  • legislation and regulations to implement the classification, import/export controls and tracking requirements and environmentally sound management,
  • procedures to administer and enforce the import/export regime, and
  • support for technical initiatives and regular reporting.[80]

In signing these international agreements, Canada made a commitment to develop national legislation to promote the environmentally sound management of hazardous wastes and hazardous recyclable materials. This led to the development of the Export and Import of Hazardous Wastes Regulations, which came into force in 1992, under the former CEPA. These were replaced by the Export and Import of Hazardous Waste and Hazardous Recyclable Material Regulations (EIHWHRMR) in November 2005. Modifications were required to address new authorities with respect to hazardous wastes and hazardous recyclable materials that did not exist under the former CEPA.[81] As such, compliance with the agreements is now primarily provided through delivery of EIHWHRMR, which implements the classification requirements, as well as through regular reporting to the appropriate organization for each of the agreements.

The OECD Decision regarding PCBs requires commitments to ensure:

  • the end of all manufacturing, trade or sale of new PCBs and new PCB-containing equipment,
  • the safe use and gradual phase-out of existing PCB-containing equipment and
  • the safe handling and disposal of waste PCBs and PCB-containing products.[82]

These issues are addressed by the PCB Regulations, the PCB Waste Export Regulations and the Federal Mobile PCB Treatment and Destruction Regulations under CEPA 1999.

The need for Canada to meet its international commitments in a sufficiently timely fashion was a driver for repealing the Chlorobiphenyls Regulations and the Storage of PCB Material Regulations in 2008 and for replacing them with the PCB Regulations under CEPA 1999, which set specific dates for the destruction of PCBs in service and in storage.[83] Further, the PCB Waste Export Regulations, 1996, restrict exports of Polychlorinated Biphenyl wastes to the United States for treatment and destruction. Export is only allowed for thermal or chemical destruction at authorized and environmentally sound United States facilities. Export for landfilling and purposes other than those listed in the regulations is not allowed and exports of PCB wastes to countries other than the United States continue to be prohibited.”[84]

Program staff who were interviewed for the evaluation were also of the view that EC’s work under the WRM SSA has contributed to meeting Canada’s international commitments with respect to harmful substances and waste. Staff stated that EC meets its reporting requirements internationally by collecting data from industry and by participating in international meetings that address this topic. Program staff also noted that regulatees need to have a clear understanding of reporting requirements and their link to international reporting as well as respecting reporting timelines in order for Canada to meet its international reporting requirements.

External stakeholders believe that EC has shown leadership internationally and is seen as being proactive on the international stage. Survey respondents were also of the view that EC was doing well on this outcome, with 92% of survey respondents[85] indicating that EC’s waste reduction and management activities have contributed to Canada meeting its international commitments with respect to harmful substances “somewhat” (60%) or “to a large extent” (32%).

Intermediate outcomes:

a) Deposits of harmful substances in waters frequented by fish are minimized

The majority of interviewees believe that EC’s WRM activities are supporting the achievement of this outcome. In general, program staff are more positive about the achievement of this outcome than government /external stakeholders. Government stakeholders speaking about the MMER stated that it was too early to comment on whether success has been achieved in this area, and external stakeholders held mixed views on whether progress had been made. Jurisdictional issues play an important role in the achievement of this outcome, specifically the need for coordination/harmonization of regulations across provinces and/or among levels of government.

In the case of the MMER, external stakeholders interviewed believe there has not been a decrease in environmental damage because of TIAs, which they believe makes it less expensive to dump tailings than to build containment structures. MPD notes, however, that TIAs are only authorized after it has been proven during an environmental assessment that the TIA is the best overall environmental, technical, economic and social alternative.

Survey respondents provided favourable ratings[86] for EC’s contribution to the achievement of this outcome, with 92% of survey respondents rating that “releases of harmful substances into water frequented by fish are minimized” “somewhat” (49%) or “to a large extent” (43%). Questions were asked in regard to EC’s contribution in general, rather than by regulation.

Compliance results, as reported by the Enforcement Branch (see Table 10), are high for all regulations, ranging from 81-100% between 2007-2008 and 2010-2011 based on a targeted sample of regulatees. Compliance results are also high for the MMER (between 2003-2009[87]) and PPER (in 2008[88]), based on self-reported data of the entire regulated community to the Regulated Information Submissions System, ranging from 93-100%. Compliance results for the storage tank regulations were also high (~ 85%) as identitifed in 2010 through the FIRSTS database.[89]

b) Pollution prevention measures are implemented for the deposit of harmful substances into waters frequented by fish

Overall, interviewees believe that EC’s WRM activities are contributing to the achievement of this outcome. Program staff rated this outcome more highly than government/external stakeholders, citing a variety of reasons, including that the regulations could be stricter (MMER) and the long process required to establish regulations (WSER). Program staff noted the importance of compliance promotion in achieving this outcome.

Government/external stakeholders believe that provincial activities make an important contribution to this outcome and emphasize the continued need for collaboration with the CCME/provinces.

Survey respondents also provided high ratings[90] regarding the achievement of this outcome, with 93% of survey respondents noting that EC had contributed “somewhat” (47%) or “to a large extent” (46%) to pollution prevention measures being in place to minimize the release of harmful substances into water.

c) Reduced releases of harmful substances into the environment[91]

Compliance results, as reported by the Enforcement Branch (see Table 10), are high for all WRMD regulations, ranging from 87-100% between 2007-2008 and 2010-2011 based on a targeted sample of regulatees. Additionally, all stakeholders interviewed agree that positive progress is being made on this outcome, particularly given that work in this area is conducted with limited resources. Survey respondents also had a positive assessment[92] of progress in this area, with 92% of survey respondents indicating that EC’s WRM activities had contributed “somewhat” (55%) or “to a large extent” (37%) to reducing releases of harmful substances into the environment.

The fall 2009 Auditor General report identified a gap with respect to delivery of this outcome in First Nations communities, noting that EC and AANDC need to work together to address environmental gaps on reserves.[93] This is being addressed through the work of the Federal House, and in part through the storage tank regulations.

Final outcome:

a) Reduced threats to the environment from harmful substances

Interviewees believe that progress has been made on the achievement of this outcome. Despite the progress made, however, there are indications that a lack of sufficient resources may hinder further progress, particularly for WRMD. Other factors that may limit progress on this outcome include the lack of clarity in the governance of the FA-PPP as well as capacity issues with First Nations communities to implement regulations (e.g., storage tank regulations, upcoming WSER requirements).

Consistent with the findings related to medium‑term outcomes, 92% of survey respondents indicated[94] that EC had contributed “somewhat” (55%) or “to a large extent” (37%) to reducing threats to the environment from harmful substances.

The following tables provide a summary of the survey responses regarding achievement of outcomes. Table 10 displays the results of the WRM survey. It lists the responses to each question as well as the total number of respondents per question. Results are presented using the negative to positive rating scale (not at all, somewhat, to a large extent) employed to measure responses to each question.

Table 10: Survey Responses on Achievement of Outcomes[95]
 To what extent have Environment Canada’s waste reduction and management activities contributed to the following?Not at allSomewhatTo a large extentTotal (n)
Responsibility for legislative and regulatory activity across different jurisdictions is clear (i.e., provincial/territorial/federal).26%53%21%80
Legislative and regulatory activity is streamlined across different jurisdictions.43%43%14%79
Compliance with regulatory and non-regulatory requirements is increasing.6%43%51%72
Practices for the handling, disposal and recycling of end‑of‑life products and substances are improving.15%51%34%65
Practices for the movement of harmful substances across Canadian or provincial/territorial boundaries are improving.17%54%28%46
Canada is meeting its international commitments with respect to harmful substances and waste.9%62%30%47
Releases of harmful substances into water frequented by fish are minimized.8%49%43%75
Pollution prevention measures are in place to minimize the release of harmful substances into water.7%47%46%81
Releases of harmful substances into the environment are reduced.7%55%37%83
Threats to the environment from harmful substances are reduced.9%55%37%82

 

External factors which impact the achievement of outcomes

Jurisdictional issues due to a lack of clarity with respect to roles and responsibilities are seen by interviewees as the key factors that limit the achievement of the outcomes. In particular, government stakeholders identified the lack of clarity in roles between EC and DFO with respect to the FA. These stakeholders stated that this was impacting progress on the FA-PPP and that work was needed to enhance communication, coordination and reporting to ensure a consistent approach is taken. External stakeholders echoed this opinion and cited a lack of consistency among jurisdictional requirements, particularly for those operating in multiple jurisdictions, as hampering the achievement of environmental results (FAAD and WRMD). In SWMD’s wastewater section, jurisdictional issues with AANDC and First Nations communities play a role in limiting achievement of environmental outcomes, as there is not always clear agreement on responsibilities.

Industry Canada’s Infrastructure Program was also identified as an external factor impacting the achievement of outcomes. This program was identified as having had a positive impact on the outcomes, as it provided financial support to municipalities to upgrade their water and wastewater treatment facilities, thus reducing the release of harmful substances into water from this source.

Program stakeholders repeatedly identified insufficient resources to carry out all core activities as a challenge. This viewpoint was reinforced in the document review, which also revealed several instances where priority areas could not be addressed due to resourcing issues. These include work to develop wastewater performance standards for the North, reviewing FA water quality risks for agriculture, engagement with DFO on risk management for aquaculture, a review of dated regulations as committed to in the CESD spring 2009 audit, and upgrading the online reporting tool for new PCB Regulations.[96] Capacity issues were also present outside the SSA, as issues were identified with partners, both within the Department (e.g., S&T Branch, Chief Information Officer Branch[97] (CIOB)) and external to it (e.g., provinces and territories), which directly impact the work of the WRM SSA.

Performance Data

Evaluation Issue: PerformanceIndicator(s)MethodsRating
5. Are appropriate performance data being collected, captured and safeguarded?If so, is this information being used to inform senior management / decision makers?
  • Existence of effective performance measurement strategy
  • Extent to which performance data inform/support EC’s decision-making processes
  • Document Review
  • Key Informant Interviews
Progress Made; Attention Needed

Appropriate information is being collected and used for decision making, although the need to provide more comprehensive information to support decision making was identified in WRMD, as was the need to continue ongoing efforts to replace existing manual monitoring and reporting systems with online capability for FA regulations and CEPA regulations under WRMD. Additionally, more could be done to understand the comprehensive performance story beyond each individual regulation, as well as to understand the overall compliance results for each regulation. In the case of the relatively few G&Cs managed within the SSA, insufficient performance information was available to be able to report on their overall performance.

Performance Measurement Framework and Strategies

EC’s 2011-2012 Performance Measurement Framework includes expected results, outputs and indicators for the WRM SSA (3.1.2.1), as well as for the higher‑level SA (3.1.2) for Waste Management. However, as shown in Table 11 below, the information is incomplete, as the outputs presented pertain only to WRMD and do not represent the outputs of the other divisions which support WRM activities. Data has yet to be reported against these indicators.

Table 11: Excerpt from EC’s 2011-2012 Performance Measurement Framework
PAA Ref. # Strategic Outcome / Expected Result / OutputPerformance IndicatorFrequency of Data CollectionData SourceTargetDate to Achieve Target
3.1.2Expected ResultReduced releases to the environment of toxic substances and other substances of concern in wasteVolume of releases to the environment of toxic substances and other substances of concern from wasteTo be determinedNational Pollutant Release InventoryTo be determined once specific substances are chosen and baseline values are measuredTo be determined
3.1.2.1Expected ResultReduced releases of toxic substances from the industrial waste management and waste generating sectorsPercentage of waste management and industrial facilities with releases within regulated limitsAnnualReporting by facilities as required by regulations95%Varies with each regulation
3.1.2.1OutputRisk management instruments related to waste discharges and end-of-life substances of concernNumber of regulations and instruments publishedAnnualCanada Gazette2 per year2011
3.1.2.1OutputTransboundary permitsPercentage of notices received by EC where the data are entered and initial processing is completed within 15 working daysAnnualCNMTS90%2011

Although no evidence was found of a comprehensive performance measurement strategy that addresses the overall WRM SSA, performance measurement frameworks or plans did exist for some individual components of the SSA’s work. For instance, a performance measurement framework is currently under development for the FA-PPP and a logic model with associated outcomes and outputs has been prepared.[98] Additionally, a logic model that addresses the work of a program within WRMD has also been developed.[99] Although this does not include the work of Federal House, which was developed prior to the reorganization that saw the Federal House become part of WRMD, a performance measurement plan and logic model were also developed for the storage tank regulations.[100] A performance measurement and evaluation plan (PMEP) has also been prepared for the proposed WSER, which included development of a logic model and the indicators planned for measurement.[101]

Collection and Use of Performance Data in Decision Making

Although data, collected as part of the Department’s Performance Measurement Framework, are not yet available or complete, a considerable amount of data are collected related to each of the individual regulations administered by the WRM SSA.

Interviewees from FAAD, MPD and WRMD all indicated that they are collecting performance data which are used to support decision making, although some WRMD program staff indicated that, with the exception of the Storage Tank Program, data are generally inadequate to support effective decision making and the division often must rely on extrapolating outdated data. In SWMD, where the wastewater regulations have not yet been implemented, interviewees reported that data will be available after implementation, as outlined in the PMEP for the regulation.

In addition to the reporting data collected as required by the regulations, the Enforcement Branch collects data relative to the FA-PPP and CEPA 1999 regulations relative to inspections, investigations and infractions. These enforcement data are contained within the NEMISIS database.

Additional data relative to all four divisions are also collected by the Compliance Promotion and Analysis Division. Interviewees representing compliance promotion indicate that a new database has been developed to include such information as those participating in information sessions, those who were sent communication, etc. The database, which will be fully implemented in the coming fiscal year, will work with NEMISIS to provide more robust performance measurement information and is anticipated to be a better tool to support decision making.

Additionally, as noted previously, the EEM Program prepares a report every 2-3 years, which includes an assessment by regulation and is used to make recommendations to senior management on the effectiveness of the regulations.

Online Reporting

The evaluation found that, although there are many sources of performance data, there are many challenges related to the online capabilities for performance data, which has impacts on efficiency and the availability of reporting based on the data, as evidenced below.

  • NEEMO has faced significant challenges related to the fact that their electronic reporting system is no longer available. Previous electronic reporting supported by the EEM Program was determined to be inconsistent with the Treasury Board Secretariat’s Common Look and Feel Standards for the Internet and was removed until a new system could be developed. An interim measure requiring NEEMO staff to manually input data provided by regulatees has been in place since 2009. The objective is to develop a new electronic EEM reporting system common for regulations under FA-PPP(PPER, MMER), but this new system is not yet in place. The project, which is being undertaken concurrent with the development of an electronic reporting tool for WSER, is in the business analysis phase and is not expected to be implemented before 2012-2013.
  • FIRSTS has yet to have many of its functions completed, due to lack of programming time from CIOB.[102]
  • Permitting and reporting of hazardous waste and hazardous recyclable material shipments is still paper-based. CNMTS (for the EIHWHRMR) is currently in year two of a four-year modernization project to support real-time, online tracking and web-based access for regulatees. While this is a four-year initiative, funding is only approved on a yearly basis and there is concern among program staff that the resources to allow the program and CIOB to complete this could be at risk, given the current government‑wide fiscal restraint associated with the need to find $4 billion in annual savings within the public service, as announced in the 2011-2012 Budget.[103]
  • Data from the regulated community relevant to PCBs are also currently entered manually. An online reporting tool was scheduled for implementation in March 2010; however, the document review identified this as an unfunded area for 2009-2010.[104]
Reporting of Compliance Results

The evaluation also identified challenges related to the definition of regulatory compliance and a lack of clarity with respect to what individual compliance results represent. Different sources report compliance results from the context of different subsets of regulatees. For example, the Enforcement Branch will look at compliance results among those subject to an inspection, who may be higher‑risk sub-segments of the regulatee population, whereas RISS data provide regulated self-reported compliance data from all regulatees. The evaluation found several instances where different compliance results were identified for the same regulation, without a clear identification of what was being measured or its implications for risk management.

This challenge is not unique to the WRM SSA. The Enforcement Branch and Compliance Promotion and Analysis Division at EC are currently contributing to an international project on compliance rates with the OECD, which resulted in a working paper published in 2010 that looked at performance measurement related to environmental compliance issues.[105] The report noted that compliance rates are a key outcome indicator and found similar issues to those identified in this evaluation, including a large number of approaches to measure compliance rates and many different definitions of what is meant by compliance.

Assessment of Grants and Contributions

Information on the G&Cs funded under the WRM SSA was difficult to obtain; files are maintained within the various WRM divisions by different financial and project officers. Nevertheless, a file review was conducted of eight WRM SSA G&Cs supported between 2008 and 2010 for a total of $211,000. The G&C funding was primarily used to fund international projects as part of Canada's international obligations under the Basel Convention, as well as work undertaken by the OECD's Working Group on Waste Prevention and Recycling.

The file review revealed that the quality of expected outcomes was quite mixed. In some cases, outcomes were clear and well-aligned to overall project objectives, while in other cases expected outcomes were not provided or were difficult to discern.

The file review was unable to assess the actual performance of the projects due to the fact that expected performance indicators were either not provided or were not clearly aligned to program expected outcomes, and in all cases final reports presenting project results were not available.

Program Design

Evaluation Issue: PerformanceIndicator(s)MethodsRating
6. Is the program design appropriate forachieving expected program results?
  • Clearly defined and understood governance structure, including program processes, roles, responsibilities and accountabilities
  • Program resources/capacity commensurate with expected program results
  • Views on the appropriateness of program activities, processes and governance structures
  • Document Review
  • Key Informant Interviews
  • Online Survey
  • File Review
Progress Made; Attention Needed

Generally speaking, the governance and priority‑setting mechanisms are viewed as appropriate, although resource shortages and high turnover have contributed to a lack of clarity in the governance processes for WRMD and there appears to be a lack of engagement in the priority‑setting process for both SWMD’s wastewater section and WRMD. Additionally, there is a recognized need to improve the clarity of the roles and responsibilities among those involved in the delivery of WRM SSA activities. Collaboration and communication with stakeholders is an important part of the work of the WRM SSA and is generally working well. Resource shortages are a challenge facing the WRM SSA; the divisions are managing within their current levels through prioritization, but certain key activities are not taking place because of resource shortfalls, and WRMD in particular is unable to deliver on its commitments in a timely manner due to funding and human resource gaps, which has resulted in dissatisfaction among stakeholders.

Appropriateness of Governance and Priority Setting

Priority setting occurs to guide both the program activity of WRM SSA divisions, and compliance promotion and enforcement activities. Each division within PRSD prepares annual planning and budgeting documents that provide a list of plans and priorities, expected results/deliverables and project work associated with carrying out these plans and, in the case of FAAD and WRMD workplan documents, rankings identifying areas of priority for compliance promotion. An annual workplan for PRSD is also developed which represents the work of the SSA as a whole. The Compliance Promotion and Analysis Division and the Enforcement Branch also develop their own planning documents which show linkages to government priorities, as well as an assessment of compliance promotion and enforcement risk.

Program staff and government stakeholders gave mixed reviews on the appropriateness of program priority‑setting processes and mechanisms. In general, there is satisfaction with the priority-setting and governance mechanisms with respect to FAAD and MPD. In these divisions, interviewees indicated that the process is clearly defined and understood and for the most part works well. Compliance promotion staff also expressed satisfaction with their overall priority-setting process and believed that they had appropriate input from programs into the process. For SWMD’s wastewater section and WRMD, however, concerns were expressed regarding the level of engagement in the priority-setting process at the working level, with a general sense that priority setting was top-down. Additionally, WRMD program staff believed that the governance processes related to resource allocation were not clear and noted that organizational changes and management turnover have also contributed to a lack of clarity regarding governance processes.

At a more general level, concerns were expressed related to governance in support of FAAD’s FA-PPP focal point role -- a role which was established in response to the CESD audit. The CESD spring 2009 audit found that EC had not clearly identified what it had to do to fulfill its responsibility for the FA-PPP and had not established clear priorities or expected results for the administration of the prohibition. The audit included a recommendation (Recommendation 1.93) that EC set out clear objectives and results expectations for its FA responsibilities and establish accountability for achieving the desired results, including providing national coordination and guidance on the administration of the Act. As a result, EC committed to producing a Results-Based Management and Accountability Framework (RMAF) in 2009–2010 for EC’s FA responsibilities that would clearly identify the objectives, responsibilities and expected results, including how national coordination and guidance on EC’s administration of the Act will be provided. As of the spring 2011 update to the CESD, EC was in the midst of preparations for the implementation stage for this recommendation, having completed a mapping exercise, draft logic model and a review of strengths and gaps.

Clarity of Roles and Responsibilities

The survey results found that those involved in delivery of a regulation or regulatory instrument (n=20) gave relatively low ratings regarding appropriateness and clarity of roles and responsibilities, with only 50% either somewhat or strongly agreeing that the processes are appropriate to work together, 45% in agreement that communication is appropriate, and 60% in agreement that roles and responsibilities are clearly defined and understood.

This is consistent with previous findings from the CESD spring 2009 audit, which identified a need to clarify the relationship between DFO and EC for the FA-PPP and included a recommendation that DFO, with the support of EC, should clarify EC’s role, including the expected interactions between the two departments to support the delivery of the 1986 Habitat Policy. In response to the CESD recommendation, both departments agreed and committed to review the administration of section 36 of the FA (by March 31, 2011) and to renew the MOU (by March 31, 2012). As of the spring 2011 update to the CESD audit, EC was in the planning stage for this recommendation. Discussions have been put on hold pending EC and DFO senior management discussions of roles and responsibilities related to the FA-PPP.

The evaluation also found that there are gray areas associated with the FA which result in a lack of clarity for the roles of DFO and EC, including in the area of aquaculture. DFO has responsibility for the fisheries and habitat-related aspects of the FA, and is the established federal lead for the aquaculture sector, since aquaculture is a fishery, as ruled by the British Columbia Supreme Court in February 2009. Meanwhile, EC is responsible for the administration and enforcement of the pollution prevention provisions of the FA, which apply generally to all sources and sectors, including aquaculture. There is also overlap with respect to the application of the general prohibitions related to the FA-PPP, and both departments sometimes respond to these situations for enforcement.

Collaboration and Communication among Stakeholders

The evaluation found considerable evidence of instances of collaboration and extensive consultation in the work performed under the WRM SSA. Within the SSA, communication and collaboration among the divisions dealing with regulations under the FA occurs on a regular basis. Networking, exchanges of information, and meetings occur as part of day-to-day work, and include such examples as FAAD working with SWMD on WSER and EEM considerations, and MPD working with FAAD on FA considerations that jointly impact the MMER/PPER.

Additionally, the process for regulatory development or amendment actively engages stakeholders. Program staff and stakeholders emphasized the importance of this collaboration and reported that it was generally working well, although this is something which requires consistent attention and can always be improved upon.

Survey respondents who had participated in consultations (n=40) were also generally positive about their effectiveness (74% somewhat or strongly agreed that the consultation process was an effective way to obtain input from stakeholders). Consultations generally received high ratings for clarity of roles and expectations (71% somewhat or strongly agreed) and clarity of material provided (71%), but received slightly lower marks on the measure “input from stakeholders was appropriately considered” (52% somewhat or strongly agreed). Of note, almost all respondents who said they strongly disagreed with this statement were those whose main involvement had been with the wastewater regulations. Several respondents specifically identified that despite a long period since the consultation occurred, they were still uncertain as to whether their feedback had been addressed.

The vast majority of survey respondents either somewhat or strongly agreed that the compliance promotion and information sessions they attended were useful (93%), that the information provided was of high quality (87%), and that the sessions were well timed in order to allow organizations to take necessary action (82%).

The effectiveness of working groups/committees are perceived somewhat less positively by survey respondents who had been involved with these (n=19), as only half (52%) somewhat or strongly agree that such interaction had been effective. The reason for this was not clear, although a few respondents identified a lack of technical understanding of the issues by those leading the committees as one possible reason why not all respondents agree that the meetings were effective.

Adequacy of Resources and Capacity

There is agreement among stakeholders that resources are generally adequate for MPD and SWMD’s wastewater section to accommodate their current workloads, although it is anticipated that resources will become a greater challenge in the future in light of planned activities and environmental trends. Resources were flagged by program staff as an area of concern in terms of the Department’s ability to fund compliance promotion at levels needed to sustain high compliance results, on an ongoing basis, for the storage tank regulations, as well as for WSER, post-implementation. MPD stakeholders also noted that the regions are currently understaffed.

Resource shortfalls in FAAD have resulted in a need to prioritize the assignment of resources, with the result that some key areas, notably FA focal point responsibilities, particularly in the area of agriculture and aquaculture, are not currently being fully addressed.

Within WRMD, resources appear to be a significant challenge, as reported by program staff, government stakeholders and regulatees, and this is having an impact on the timeliness and quality of work performed by the division. Regulatees noted that processing times for notifications related to the transboundary movement of hazardous waste have increased significantly from previous levels.

Unintended Outcomes

Evaluation Issue: PerformanceIndicator(s)MethodsRating
7. Have there been any unintended (positive or negative)outcomes?
  • Presence/absence of unintended outcomes
  • Views on whether unintended outcomes occurred
  • Document Review
  • Key Informant Interviews
  • Online Survey
n/a

Some unintended outcomes associated with EC’s work in the area of WRM were identified by all four divisions. For three of the fourdivisions (FAAD, MPD, WRMD) there was an unexpected increase in public interest or awareness as a result of the Department’s efforts. This also translates into an unintended impact in terms of human resource and capacity requirements for the associated divisions.

The primary unintended outcome identified by interviewees was the increased public attention on the Department’s activities in the area of WRM, and the resulting demands on EC’s resources in responding to public interest. A number of examples of this were identified through the evaluation:

  • The level of interest in Schedule 2 of the MMER regarding TIAs was unexpected. This was raised by both program and ENGO interviewees and was also supported by findings from the media scan. Program staff also noted that the high level of interest in the topic has caught the attention of senior politicians.
  • WRMD program staff and government stakeholders also noted that the level of interest in waste issues and in the FA, generally, has been unexpected.
  • The media scan also identified significant attention on the effects of mercury on First Nations’ traditional food sources and way of life.

Other issues were identified which placed unanticipated demands on resources:

  • The high demand for and price of iron ore originating from emerging markets such as India, China and South America has resulted in significant growth in metal mining in Canada, which puts further strain on the resources of MPD (e.g., program staff noted that the number of environmental assessments per year has increased from approximately 30 to 60).
  • The Natural Resources Canada phase-out of inefficient lighting raised the need to address the associated waste implications, which program staff acknowledged had initially not been adequately considered.
  • Despite being actively involved in the development of the regulatory impact assessment statement, AANDC noted that they had not been fully aware of the financial implications associated with ensuring First Nations have the capacity to implement the storage tank regulations.

Efficiency

Evaluation Issue: PerformanceIndicator(s)MethodsRating
8. Is the program undertaking activities and deliveringproducts in the most efficient manner?
  • Views on whether the cost of producing program outputs is as low as possible
  • Views on how the efficiency of program activities could be improved
  • Views on whether there are alternative, more efficient ways of delivering program activities and outputs
  • Document Review
  • Key Informant Interviews
  • Literature Review
  • File Review
  • Financial Analysis
~ Progress Made; Attention Needed

The WRM SSA has many processes in place which support the efficient delivery of the WRM SSA’s outputs; however, some weaknesses were also identified which require attention. Governance and priority setting processes are in place which are generally viewed as appropriate, and coordination and collaborationinternally and with regulatees is working well. There were, however, issues identified in terms of the clarity of roles and responsibilities between EC and DFO, as well as opportunities for improved efficiencies with the provinces. The regulatory approach for risk management was viewed to be an effective approach and no viable alternative delivery models were identified in the evaluation.

As discussed previously, the WRM SSA has processes in place for governance and priority setting which are viewed as appropriate, for the most part, although some areas for improvement were identified relative to improved clarity in the governance process for WRMD, particularly as it pertains to resource allocation. Furthermore, extensive collaboration and consultation occurs in the work performed under the WRM SSA that is generally working well, including a process for regulatory development and amendment that actively engages stakeholders. A need to improve the clarity of the roles and responsibilities among those involved in the delivery of WRM SSA activities was identified, however, particularly as it relates to delivery of the FA-PPP with DFO, as well as a need for improvements to streamline and coordinate activity with the provinces.

When asked to what degree the WRM activities were being delivered in an efficient manner, interviewees also expressed mixed views. On the one hand, program staff from each division agreed that their activities were being delivered in the most efficient manner. On the other hand, government stakeholder interviewees were divided on the issue, noting the overlap and duplication in some areas with provincial activities (WRMD), inefficiencies stemming from failing to consider interrelated impacts between regulations when developing the WSER, and using different definitions for the same term in different regulations.

Although program staff believe the activities of each division are delivered in an efficient manner, a number of suggestions for improvements were made. These suggestions were most often focused on improved coordination with other federal government departments (to reduce overlap and duplication) and provincial governments (to reduce duplication and increase harmonization).

In 2009, in an effort to improve efficiencies, the EEM Program initiated an analysis, prepared by FAAD, to review their program. This analysis revealed there was no clear system in place to track program spending, making it difficult to determine “whether the current level of effort reflects a cost-effective approach to completing EEM activities, achieving program success, and demonstrating to Canadians good value for money.”[106] The report also found that proportional spending on the EEM Program appeared to be high. Although EEM activities constituted one part of two broad and complex regulations (PPER and MMER), two thirds of total staff and approximately one half of the operating budget related to the two regulations was spent on EEM. Learnings from this report were taken to improve program functioning. It should be noted that these type of data were not available for other regulations or aspects of the regulatory cycle.

Regulatory vs. Non-Regulatory Approach to Risk-Management

The work undertaken as part of the WRM SSA is highly regulated with a few non-regulatory instruments. Specifically, across the four divisions existing in the SSA, there are presently sixteen regulations in place (including two proposed) and three non-regulatory instruments (notices).

A review of literature focusing on risk management strategies (i.e., voluntary and non-voluntary instruments) revealed that, overall, a regulatory approach to risk management can be an effective tool to ensure achievement of environmental outcomes. In some cases, there is evidence that illustrates situations where voluntary measures have not been as effective. In other cases, the use of a regulatory approach is effective, as long as it is implemented in conjunction with other non-regulatory measures, to provide a mix of policy instruments. Important considerations to take into account when determining risk management strategies include the need to ensure adequate resources for enforcement, the cost-benefit of compliance vs. non-compliance, the importance of corporate image/reputation, the effects of “social licence” pressures (particularly from local communities and environmental activists) and effects on market competition (e.g., barriers to entry, and restrictions on competition leading to a concentration of players in the industry).[107]

In the case of WRM, there are two examples where EC has moved from voluntary to non-voluntary measures when the voluntary measures were not having the desired impact. In 2008, EC adopted a regulatory approach towards the reduction of PCB inventories after it became apparent that voluntary initiatives were not achieving results at a rate which would allow Canada to meet its international commitments in a timely fashion.[108] Also in 2008, EC adopted a regulatory approach towards the reduction of the risk of contamination of soil and groundwater from petroleum products and allied petroleum products, through the storage tank regulations, after it became apparent through two former initiatives that voluntary initiatives were not achieving desired results.[109]

Alternative Delivery Models

The evaluation did not identify any clear opportunities for alternative models of delivery. However, it was noted that, presently, a pilot project initiative led by a private company without sponsorship from AANDC or EC is working with First Nations to develop the technical skills needed to comply with the storage tank regulations. The results of this pilot project may provide interesting insights into possible future opportunities.

In addition, a suggestion was made in the context of WRMD to streamline the permit process by developing an electronic format and potentially charging for permits.

Economy
Evaluation Issue: PerformanceIndicator(s)MethodsRating
9. Is the program achieving its intended outcomes in the most economical manner?
  • Extent to which program intended outcomes have been achieved with the least possible program cost
  • Views on whether good value is being obtained with respect to the use of public funds
  • Document Review
  • Key Informant Interviews
  • File Review
  • Financial Analysis
~ Progress Made; Attention Needed

An examination of financial inputs within the WRM SSA demonstrates an increase in expenditures over the four‑year period of theevaluation of approximately 21%, while a comparison of budget versus actual expenditures shows SSA spending was under budget for three of the four years.[110] Limitations with financial information (e.g., budgeted figures were not available at the divisional level) do not support an analysis of the full cost of delivering the WRM SSA’s risk management instruments. Although some suggestions for improvement were identified, the majority of interviewees felt good value was being obtained through EC’s WRM activities.

Comparison of Budgeted and Actual Expenditures

Tables 1 and 2 present the detailed budgeted financial figures and actual expenditures related to the WRM activities. The budget was established at the SSA level, and as such, budgeted expenditures were not available by division. Table 12 below presents a summary of total actual expenditures by division and for the entire PAA element for the 2007-2008 to 2010-2011 fiscal years. The divisions included are FAAD, MPD, APSD, Sustainable Water Management, Federal House and WRMD.

Table 12: Total Actual Expenditures by Division and PAA Element
Division
2007-20082008-20092009-20102010-2011
Source: data supplied by Finance Branch (in April and May 2011), showing actual expenditures in the fiscal years shown for fiscal codes linked to the WRM SSA
FAAD
$2,977,394
$3,608,706
$3,161,394
$3,710,711
MPD
$1,181,151
$1,427,544
$1,657,025
$1,730,973
APSD
$3,915,343
$2,744,099
$2,908,655
$3,794,610
Sustainable Water Management
$2,579,978
$1,655,262
$1,811,171
$2,120,618
Federal House
$1,335,365
$1,088,837
$1,097,484
$1,673,992
WRMD
$2,850,420
$4,137,815
$3,827,454
$4,003,011
Total for SSA
$10,924,308
$11,918,164
$11,554,528
$13,239,305

Overall, actual expenditures for the SSA have increased over the evaluation period (2007-2008 to 2010-2011) from $10,924,308 to $13,239,305 (an increase of approximately 21%). The largest increase in actual expenditures occurred in WRMD, increasing by $1,152,591 (an increase of 40.4%).

Table 13, below, provides a summary of budgeted vs. actual expenditures for the WRM SSA for the period of the evaluation and demonstrates that between 2007-2008 and 2009-2010, SSA spending was under budget. No clear explanation for the variances was available, but program staff believed that the issue was likely related to the fact that budgets are managed by line items, and could be the result of a coding issue whereby managers who work on projects under more than one PAA element may be coding expenditures to different program elements. Additionally, given EC’s resource allocation processes, it is unclear whether this money was available to the SSA at year end or the funds had been reallocated elsewhere.

Table 13: WRM SSA – Budgeted vs. Actual Expenditures
 2007-20082008-20092009-20102010-2011
Source: data supplied by Finance Branch in April and May 2011
Budgeted$11,332,565$13,213,584$12,942,278$12,679,778
Actual$10,924,308$11,918,164$11,554,528$13,239,305
Variance$408,257$1,295,420$1,387,750-$559,527
% Variance4%10%11%-4%

WRM SSA activities are also supported by the Compliance Promotion and Analysis Division and the Enforcement Branch, whose budgets are not included within the figures presented in Tables 12 and 13. While the WRM SSA addresses eight regulations under CEPA 1999 and six regulations under the FA, these organizations are supporting all of EC’s regulatory programs, which includes over fifty regulations related to CEPA 1999, six regulations related to the FA, numerous codes of practice, guidelines, pollution prevention plan notices and environmental protection agreements, and regulations related to wildlife-related legislation.

EPOD’s compliance promotion expenditures for the Department ranged from $5–7 million during the four-year period of the evaluation, with salary representing approximately three quarters of expenditures, and O&M representing the remaining quarter.

Enforcement expenditures for the Department ranged from $37.1–$41.4 million during the period of the evaluation, with annual expenditures ranging from $23.5–$26 million for the Environmental Enforcement Directorate, which has responsibility for administration of CEPA 1999 and the FA.

Unfortunately, expenditures for these two groups are not currently tracked at the level of the individual regulations or divisions they support, so it was not possible to isolate the contribution from compliance promotion[111] and enforcement in support of the WRM SSA. This is a significant barrier in terms of being able to measure the cost to deliver these regulations and to fully understanding their value.

The federal government conducts Regulatory Impact Analysis Statements (RIASs) for all regulations and regulatory amendments. These statements, which are subject to public scrutiny through the Canada Gazette process, include sections on alternative approaches (i.e., voluntary measures, economic instruments, enacting regulations, or retaining the status quo) as well as an analysis of costs (based on future predictions) and benefits for a given regulation, although these are predicitive in nature only. For example, the RIAS that was prepared for the proposed WSER identified the issue that was to be addressed by the proposed regulations and then provided a cost-benefit statement that quantified the costs of implementing the regulations (~ $5.9 billion in discounted 2010 dollars) and compared them to the overall approximate benefits (~ $17.6 billion). This results in a benefit to cost ratio of almost 3:1. Additionally, the RIAS goes on to describe who will bear the cost of the proposed regulations and describes the benefits, which are noted to be difficult to quantify. Finally, the RIAS describes business and consumer impacts and domestic and international coordination.[112]

Interviewed program staff believed that EC’s WRM activities represent good value with respect to the use of public funds, noting a modest operating budget and demonstration of good results, although WRMD staff noted that value could be improved through leveraging of regional resources within EC and by increased partnering with municipalities. As noted earlier in the report, the WRM SSA is prioritizing based on available resources and is not able to deliver on all identified priority areas (e.g., aquaculture and agriculture; modernization of tracking systems).

The majority of government and external stakeholders interviewed also believed that good value was being obtained through EC’s WRM activities, with several identifying positive collaboration and the efforts to bring together the relevant partners and stakeholders. Several interviewees offered suggestions to improve the value provided, however, which included improving harmonization with the provinces and reducing the time and money spent on the consultation process with industry.

5.0 Conclusions

Relevance

There is a continued need for the activities in support of WRM at EC, as they contribute to the general health and safety of all Canadians as well as to environmental health and ecosystem resilience in Canada through the provision of key regulations and non-regulatory instruments in the area of water quality and waste.

There is a clear legislated mandate for federal government involvement in the provision of waste reduction and management services, set by regulations and guidelines under CEPA 1999 and the FA, as well as within the responsibilities of the Canadian Water Act. Additionally, the regulations support Canada’s commitment to the Basel Convention on the Control of Transboundary Movements of Hazardous Wastes and their Disposal.

Further, the activities of the WRM SSA provide support to the following recently articulated federal government priorities:

  • water quality, and a reduction of risks associated with wastewater effluents;
  • Northern strategy, focusing on pollution prevention and waste management activities in the North; and
  • Clean drinking water and effective wastewater management on First Nations reserves.

Performance

Progress has been made towards the achievement of the WRM SSA’s immediate and intermediate outcomes. For the most part, regulatees are aware of and understand the requirements of the FA-PPP and CEPA 1999, and EC’s WRM activities have contributed to progress in terms of regulatees meeting monitoring and reporting requirements for the management of harmful substances. Lesser results were, however, identified in terms of the clarity of monitoring and reporting requirements and the degree to which regulations are well communicated and clearly understood, for the two regulations managed by WRMD pertaining to the transboundary movement of hazardous waste, and also for the proposed WSER, recognizing that some lack of clarity regarding WSER is to be expected given that this regulation is still under development.

The data reveal generally high compliance results (typically greater than 90%) across all regulations, with the exception of the PCB Regulations for which compliance results are estimated to be less than 50%. Multiple sources are available which speak to compliance, but there is variation in terms of how compliance results are calculated and reported among these sources. Attention should be given to clarify the methods by which calculations are based and the different subsets of regulatees they represent. Using multiple lines of evidence to describe compliance is a necessary and useful goal, but failure to properly describe the sources and to reconcile these into a global assessment of progress is necessary to improve transparency and support decision making.

The evaluation also found a clear link between compliance results and compliance promotion activities, and those in the regulated community who had participated in compliance promotion and information sessions were very positive in terms of the usefulness and the quality of information provided. While the strategy of an intensive two-year focus on compliance promotion for new or revised regulations was effective in most cases, this strategy does not work as well for those regulations where there is high turnover among regulatees (noteably for regulatees on Aboriginal lands), as is the case with the storage tank regulations, and is also anticipated to be an issue for the WSER.

There was generally high recognition of the contribution of EC toward the achievement of the SSA’s wider environmental outcomes, and stakeholders indicated that progress continues to be made in this area.

For the most part, governance and priority‑setting mechanisms for the SSA are appropriate and support efficient delivery. Divisional planning and priority setting guides the activities of the PRSD divisions, and processes are in place to coordinate priorities with the Compliance Promotion and Analysis Division and the Enforcement Branch. A lack of clarity in the management processes in WRMD was identified, specifically with respect to resource allocation; organizational changes and management turnover are seen as contributing to these challenges. Further, concerns were identified regarding a lack of engagement in the priority setting for both SWMD’s wastewater section and WRMD, with a general sense that priority setting was top-down.

The evaluation, as well as the CESD audit, found a lack of clarity with respect to the division of roles and responsibilities with DFO for delivery of the FA-PPP. There also appears to be several areas where the roles and responsibilities among the various jurisdictions and organizations involved was unclear, particularly for SWMD with respect to the wastewater regulations and WRMD regarding the EIHWHRMR.

The issue regarding the ambiguity with respect to delivery of the FA-PPP was also identified in the CESD spring 2009 audit report. This report identified concerns with the roles and responsibilities between EC and DFO, and recommended the renewal of a dated MOU (1985) between the two departments which aims to lay out the mutual responsibilities with respect to the administration and enforcement of the FA-PPP. The current evaluation also identified the outdated MOU between DFO and EC as a contributing factor to this lack of clarity.

As monitoring and reporting is a requirement under most regulations, there is a considerable amount of performance information, coming from a variety of sources (e.g., the Compliance Promotion and Analysis Division, the Enforcement Branch, and regulatory databases). However, there is no cohesive performance story that speaks to the overall effectiveness of EC’s WRM SSA, and its regulations as a whole, in achieving its environmental outcomes. Further, while a great deal of compliance information is available, it is difficult to interpret what each of the individual sources represents, and much of the data continue to be managed manually.

G&Cs are an exception to the relatively positive performance measurement story. While relatively few G&Cs exist within the SSA (between $37,000 and $90,000 per year), there was a lack of available performance information for them, resulting in an inability to report on the G&Cs’ overall performance relative to the SSA outcomes.

Further, in spite of the seemingly high compliance results for the regulations assessed within this SSA, the evaluation encountered difficulty determining the efficiency and economy of the regulations, given the challenges associated with tracking/measuring resource allocations by individual regulation, including quantifying the level of support to the various regulations from departmental support functions (e.g., the Compliance Promotion and Analysis Division, the Enforcement Branch), the regions (e.g., EPOD), and within PRSD divisions. Another element which contributed to this difficulty was the lack of budgeted financial information at the divisional level to compare forecasted versus actual expenditures for the PRSD divisions.

Interviews and documentation suggest that resource shortages constitute a challenge to realizing EC’s WRM activities. Although the divisions are managing with their current levels through prioritization, several key activities are not taking place, including activity related to delivering on the recommendations from the CESD spring 2009 audit, and conducting work in the area of aquaculture and agriculture. Resources were also identified by stakeholders as having an impact on the delivery of key activities, including replacing manual reporting systems with online capability, meeting the high regulatory demand in the regions associated with environmental assessments for MPD, and the ability of WRMD to meet demand for export notifications in a timely manner. Furthermore, resources were flagged by stakeholders as an area of concern in terms of the Department’s ability to fund compliance promotion at levels needed to sustain high compliance results on an ongoing basis for the storage tank regulations, as well as for WSER, post-implementation.

6.0 Recommendations

The following recommendations are based on the findings and conclusions of the evaluation. The evaluation recommendations are directed to the ADM-ESB, who is the senior departmental manager responsible for the overall management of this area.

Recommendation 1: Revise and update the management actions as committed to in response to the CESD spring 2009 audit recommendations to ensure that they can be implemented.[113]

The evaluation independently replicated many of the conclusions drawn by the CESD. In particular, the evaluation identified a need for high-level performance measurement data, a review of dated regulations, and greater clarity of roles and responsibilities between EC and DFO related to the FA-PPP. These findings echo CESD findings related to identifying and tracking performance measurement information (1.93), the review of several dated regulations and guidelines (1.120), and clarification of roles and responsibilities between DFO and EC related to the FA-PPP (1.134). EC accepted the recommendations of the 2009 CESD report and identified actions to address the issues. While EC has made some progress, commitments have not been met within identified timelines.

Recommendation 2:  Conduct a review and develop action plans to strengthen internal WRMD processes related to improving the clarity and communication of:

i.WRMD priority setting and resource allocation decisions; ii. reporting requirements for both the EIHWHRMR and the Interprovincial Movement of Hazardous Waste Regulations; and iii. roles and responsibilities among various jurisdictions and organizations involved in the EIHWHRMR and the PCB Regulations.

While WRMD represents an important component of the WRM SSA, the division appears to face a number of challenges impacting its ability to achieve outcomes. Many of these issues relate to a need for more clearly defined processes and improved communication to stakeholders, including in relation to internal operations, regulatory reporting requirements, and roles and responsibilities across jurisdictions.

WRMD priority-setting processes were identified as an area for improvement, as staff believed that engagement at the working level was limited and resource allocation decisions were not clear. 

External communications were also identified as a challenge for WRMD. Evidence identified a lack of clarity both in terms of the communication of the EIHWHRMR and the Interprovincial Movement of Hazardous Waste Regulations and their associated reporting requirements. Program staff noted that in order for international commitments to be met, regulatees need to clearly understand reporting requirements. 

Finally, external stakeholders expressed a desire for greater harmonization with the provinces, both in relation to the EIHWHRMR and the PCB Regulations, and believed that a lack of consistency among jurisdictional requirements was hampering the achievement of environmental results of the EIHWHRMR.

Recommendation 3: Ensure performance measurement data are collected and are readily available to communicate the performance story and support decision making, by i) developing and populating a performance measurement strategy for the SSA (or for its two sub-components); and ii) ensuring that all online reporting and data collection tools are fully functional.

Little strategic performance information is available that speaks to the overall effectiveness of the regulations or the SSA as a whole in achieving the environmental outcomes of the SSA and higher‑level SA, although a great deal of performance information is being collected at the regulation level.

Work has begun in the form of separate logic models for the FA-PPP and a component of WRMD’s responsibilities, the waste reduction and management program, as well as through the development of a logic model for this evaluation. However, a coordinated approach to measuring against these outcomes is not yet occurring. In addition, PMEPs are developed for all regulations that triage high (e.g., the WSER). The performance measurement strategy should build upon and consolidate these existing strategies and should address the SSA’s sub-components, water quality and waste, including toxic substances. In doing this it would address multiple performance measurement requirements, reduce duplication and reporting burden, and respond to departmental requirements. It would also include provisions to ensure appropriate performance information is collected for G&Cs spending.

There is a great deal of variability in terms of the format in which reporting takes place, with a mix of electronic and paper-based reporting. In order to ensure data are readily available, the SSA should work with the Information Management Directorate[114] within Corporate Services Branch (CSB) to ensure all online reporting and data collection tools (e.g., EEM, FIRSTS, CNMTS, PCB database) are fully functional. This will also improve the consistency and efficiency with which monitoring and reporting takes place.

7.0 Management Response

Recommendation 1: Revise and update the management actions as committed to in response to the CESD spring 2009 audit recommendations to ensure that they can be implemented.[115]

ESB agrees with this recommendation. EC accepted the recommendations of the CESD report and developed a “CESD Management Action Plan” to address the issues (attached as Annex 3). While EC has made some progress, limited resources and the need for ongoing broader discussions between EC and DFO relevant to delivering on EC’s CESD commitments have meant that the Department has not delivered all of the milestones within the timelines committed to in response to the May 2009 CESD report.

Management Response – Recommendation 1
The ADM-ESB agrees with the recommendation.
Management Action
ESB has a management action plan to deliver on its response to the CESD audit, is implementing the plan, and has a process to report on progress. EC’s CESD Management Action Plan Spring 2011 Update is attached as Annex 3.
TimelineDeliverable(s)Responsible Party

Recommendation 2: Conduct a review and develop action plans to strengthen internal WRMD processes related to improving the clarity and communication of:

i. WRMD priority setting and resource allocation decisions

ESB agrees with this recommendation, but does not see it as a high-priority item relative to other issues the program needs to address. Priority setting and related resource allocation decisions are often difficult to fully comprehend, particularly at the working level, given the multitude of competing considerations and changing circumstances. In order to improve the transparency of these decisions within WRMD, quarterly division meetings will be instituted. A standing item on the agenda for these meetings will be resource allocations to the various files managed by the division.

ii. Reporting requirements for both the EIHWHRMR and the Interprovincial Movement of Hazardous Waste Regulations

ESB agrees with this recommendation. WRMD already puts considerable emphasis on communicating regulatory obligations to affected parties. Informal discussions occur daily between the division’s notification and manifest officers and regulatees on specific permit and movement issues. In addition, over the past two years WRMD has published a number of training and outreach documents on the Department’s website. The program will improve the clarity of reporting requirements and will reduce the administrative burden associated with those requirements through two planned initiatives: the planned amendments to the regulations and, most significantly, the work under way to move the CNMTS to an online application and reporting system.

iii. Roles and responsibilities among various jurisdictions and organizations involved in the EIHWHRMR and the PCB Regulations

ESB agrees with this recommendation. WRMD will address this recommendation through the planned amendments to consolidate these two regulations together with the Interprovincial Movement of Hazardous Waste Regulations. Among other things, the proposed amendments are expected to:

  • reduce the administrative burden imposed on regulatees, provinces and territories;
  • modify the regulatory reporting requirements to be compatible with the electronic submission of information; and
  • clarify the definitions of hazardous waste and hazardous recyclable material to improve enforceability.
Management Response – Recommendation 2
The ADM-ESB agrees with the recommendation.
Management Action
To address the three elements of the recommendation, ESB proposes the three actions outlined below.
TimelineDeliverable(s)Responsible Party
ImmediateTo respond to Recommendation 2i, WRMD will hold quarterly division meetings to improve the transparency of resource allocation decisions.Director, WRMD
Ongoing, with expected completion date of March 31, 2014To respond to Recommendation 2ii, the program will put the CNMTS online.Director, WRMD, contingent on resources and delivery by CSB
Ongoing, with expected completion date of December 31, 2012To implement Recommendation 2iii, the program will propose amendments to the three waste regulations, which are expected to improve the implementation and administration of these regulations, including clarifying roles.Director, WRMD

Recommendation 3: Ensure performance measurement data are collected and are readily available to communicate the performance story and support decision making by: i) developing and populating a performance measurement strategy for the SSA (or for its two sub-components); and ii) ensuring all online reporting and data collection tools are fully functional.

ESB agrees with this recommendation. After the period on which this evaluation focused, PRSD contributed refinements related to the desired outcomes and appropriate indicators for this SSA as part of the Department’s recent efforts to improve its departmental Performance Measurement Framework for 2012-2013. PRSD will continue to support the Department’s ongoing work to further refine the Performance Measurement Framework, including for the WRM SSA.

PRSD also recognizes the need to develop performance measurement strategies for the two sub-components of SSA 3.1.2.1, as recommended by the evaluation:

  • The performance measurement strategy for the FA sub-component will be the RMAF, which is a key deliverable in the Management Action Plan developed in response to the 2009 CESD audit (see Annex 3).
  • The waste sub-component will develop a performance measurement strategy that is built on the program’s existing logic model and its quarterly and annual reports on permitting activity and transboundary movements.

With respect to ensuring that performance measurement data are collected and used to communicate performance measurement, the program’s top priority is to complete the ongoing work to develop online reporting and data collection tools. These tools will provide information which will help PRSD to communicate performance and support informed decision making. Progress in this area will be contingent upon the continued availability of funding and departmental capacity to develop and implement these tools. As such, no time-bound commitment will be made to develop and implement these tools at this time.

Management Response – Recommendation 3
The ADM-ESB agrees with the recommendation.
Management Action
To address the recommendation, ESB proposes the three actions outlined below.
TimelineDeliverable(s)Responsible Party
March 2012The Performance Measurement Strategy for the FA sub-component of the 3.1.2.1 SSA is the RMAF outlined in the CESD Management Action Plan (see Annex 3).Director, FAAD
June 30, 2012The Performance Measurement Strategy for the Waste Reduction and Management sub-component of the 3.1.2.1 SSA built on the existing program logic model.Director, WRMD
OngoingWork with CSB to improve the functionality of online reporting and data collection tools.Director, WRMD, for CNMTS, FIRSTS and PCB Database, and other PRSD directors for shared databases, such as RISS and EEM. Contingent on resources and delivery by CSB.

 

Annex 1: Evaluation Issues and Questions

Annex 1 depicts the evaluation matrix, which describes the questions addressed in the evaluation. The table also provides the indicators used to measure achievement and the proposed sources and/or methods that were used to obtain evidence for each indicator.

Relevance
Does the program remain consistent with and contribute to the federal government priorities and address actual needs?
QuestionIndicatorsSources/MethodsTB Policy Issue Addressed[116]
1. Is there a continued need for the program?
  • Demonstration of societal/ environmental need
  • Document Review
Issue #1: Continued Need for Program
  • Presence/absence of other programs that complement or duplicate the objectives of the program
  • Document Review
  • Key informant Interviews
  • Reach and activities are connected to societal/ environmental needs
  • Literature Review
  • Document Review
2. Is the program aligned with federal government priorities?
  • Program’s objectives correspond to recent/current federal government priorities
  • Document Review
Issue #2: Alignment with Federal Government Priorities
  • Program’s objectives are aligned with current departmental strategic outcomes
  • Document Review
  • Views on the alignment of program objectives with recent/current federal government and departmental priorities
  • Key Informant Interviews
3. Is the program consistent with federal roles and responsibilities?
  • Program mandate aligned with federal government jurisdiction
  • Document Review
Issue #3: Consistency with Federal Roles and Responsibilities
  • Views on the appropriateness of federal involvement
  • Key Informant Interviews
Performance
Has the program achieved its intended outcomes? Are the most appropriate, efficient and economic means being used to achieve outcomes?
QuestionIndicatorsSources/MethodsTB Policy Issue Addressed
4. To what extent have intended outcomes been achieved as a result of the program?
  • Evidence of / views on intended output and outcome achievement
  • Document Review
  • File Review
  • Online Survey
Issue #4: Achievement of Intended Outcomes / Effectiveness
  • Evidence of / views on factors outside the program which have influenced the achievement of intended outcomes
  • Document Review
  • Key Informant Interviews
  • File Review
  • Views on the extent to which intended outcomes have been achieved as a result of the program
  • Key Informant Interviews
  • Online Survey
5. Are appropriate performance data being collected, captured and safeguarded? If so, is this information being used to inform senior management / decision makers?
  • Existence of effective performance measurement strategy
  • Document Review
  • Key Informant Interviews
Issue #4: Achievement of Expected Outcomes / Design and Delivery
  • Extent to which performance data inform/support EC’s decision-making processes
  • Document Review
  • Key Informant Interviews
6. Is the program design appropriate for achieving expected program results?
  • Clearly defined and understood governance structure, including program processes, roles, responsibilities and accountabilities
  • Document Review
  • Key Informant Interviews
  • File Review
  • Online Survey
Issue #4: Achievement of Expected Outcomes / Design and Delivery
  • Program resources/capacity commensurate with expected program results
  • Document Review
  • Key Informant Interviews
  • File Review
  • Views on the appropriateness of program activities, processes and governance structures
  • Key Informant Interviews
  • Online Survey
7. Have there been any unintended (positive or negative) outcomes?
  • Presence/absence of unintended outcomes
  • Document Review
Issue #4: Achievement of Intended Outcomes / Effectiveness
  • Views on whether unintended outcomes occurred
  • Key Informant Interviews
  • Online Survey

8. Is the program undertaking activities and delivering products in the most efficient manner?

  • How could the efficiency of the program’s activities be improved?
  • Are there alternative, more efficient, ways of delivering the program?
  • Comparison of program activities and products delivered by other similar programs
  • Document Review
  • Financial Analysis
  • Literature Review
  • File Review
Issue #5: Demonstration of Efficiency and Economy / Efficiency
  • Analysis of actual program operational costs in relation to the production of outputs
  • Document Review
  • Financial Analysis
  • File Review
  • Views on whether the cost of producing program outputs is as low as possible
  • Key Informant Interviews
  • Views on how the efficiency of program activities could be improved
  • Key Informant Interviews
  • Views on whether there are alternative, more efficient ways of delivering program activities and outputs
  • Key Informant Interviews
9. Is the program achieving its intended outcomes in the most economical manner?
  • Extent to which program intended outcomes have been achieved at the least possible program cost
  • Document Review
  • Financial Analysis
  • File Review
Issue #5: Demonstration of Efficiency and Economy / Economy
  • Views on whether good value is being obtained with respect to the use of public funds
  • Key Informant Interviews
  • Evidence of / views on whether there are alternative program models that would achieve the same expected outcomes at a lower-cost
  • Document Review
  • Key Informant Interviews
  • Literature Review

 

Annex 2: Summary of Findings[117]

Annex 2 provides a table displaying a summary rating for each evaluation question that was addressed, sorted by relevance, performance, and economy and efficiency. These ratings include: Achieved; Progress Made, Attention Needed; Little Progress, Priority for Attention; and Not Applicable.

Evaluation QuestionAchievedProgress Made; Attention NeededLittle Progress; Priority for AttentionNot Applicable
Relevance:
1.     Continued needX   
2.     Aligned with federal government prioritiesX   
3.     Program consistent with federal roles and responsibilitiesX   
Performance:
4.     Achievement of intended outcomes X  
5.     Appropriate performance data collected X  
6.     Program design appropriate for achieving expected program results X  
7.     Unintended outcomes   X
8.     Program undertaking activities and delivering products in the most efficient manner ~X  
9.     Program achieving its intended outcomes in the most economical manner ~X  

 

Annex 3: CESD Report on Protecting Fish Habitat May 2009 – Management Action Plan

Annex 3 depicts the Management Action Plan from the CESD Report on Protecting Fish Habitat. It lists the following: the recommendations of the report, management responses to the recommendations, progress made towards achieving the actions and milestones outlined in the management responses, the offices of primary interest to these recommendations, supporting organizations, and start and end dates for completing the prescribed actions.

Spring 2011 Update and Assessment on Progress
ParagraphRecommendationResponseAction Plan and MilestonesOffice of Primary InterestSupport OrgStartCompletion
1.93EC should set out clear objectives and results expectations for its Fisheries Act responsibilities and establish accountability for achieving the desired results, including providing national coordination and guidance on the administration of the Act.Environment Canada accepts this recommendation and will put in place a Results-based Management and Accountability Framework (RMAF) in 2009-10, for Environment Canada’s Fisheries Act responsibilities. The framework will clearly identify the objectives, responsibilities and expected results including how national coordination and guidance on Environment Canada’s administration of the Act will be provided.

Completed

  • Establish Fisheries Act (FA) RMAF Project Team in May 2009
  • Launch dialogue with DFO on EC FA S36 objectives and DFO expectations to ensure consistency between RMAF and MOU in June 2009
  • EC Discussion Document with Preliminary RMAF model, October 2010
    • Phase 1: Mapping Exercise: Departmental Profile & Governance Structure; Draft Logic Model; Review of Strengths and Gaps
  • Consultation with EC FA Network in September 2009 / October 2010

Ongoing

  • Consultation with EC Network and DFO
  • Phase 2
    • Further develop logic model
    • Senior Management Approval of Logic Model and support for forward direction for FA-PPP Logic Model implementation

Upcoming

  • EP Board review of RMAF model
  • Phase 3
    • Performance Measurement development
    • Final RMAF

Assessment 2011:
Level 1: No progress or insignificant progress
Level 2: Planning stage
Level 3: Preparations for implementation
Level 4: Substantial implementation
Level 5: Full implementation Obsolete

Public and Resource Sectors Directorate (PRSD)/ Environmental Stewardship Branch (ESB)Environmental Stewardship, Science and Technology, Enforcement, Finance and Corporate, and Audit and Evaluation Branches20092012
1.112EC should develop a risk-based approach to the Fisheries Act to identify, assess and address significant risks of non-compliance with the Act. As part of this approach, EC should determine whether there are significant Fisheries Act non-compliance issues that are not being addressed by the combination of its own administration and enforcement of the Act, and the administration of other federal and provincial legislation.Environment Canada accepts this recommendation and has assigned responsibility to the Public and Resources Sectors Directorate of the Environmental Stewardship Branch to coordinate risk management and compliance promotion priorities for Fisheries Act 36(3) and associated regulations. In 2009-10, Environment Canada will develop a work plan to identify current risks and risk management activities in non-regulated sectors, including Fisheries Act compliance promotion activities and other federal and provincial legislation, and in 2010-11, the department will complete the review of risks and risk management activities, and will adjust departmental work plans if and as may be required.

Completed

  • Consultation with EC Fisheries Act Network during the internal workshop in October 2010.
  • Establish project team for review of risks and risk management activities

Ongoing

  • Draft work plan (and resource requirements) to identify, evaluate the significance and prioritize risks in non-regulated sectors

Upcoming

  • Develop preliminary review of unregulated sectors and activities posing potential risks under FA section 36
  • Draft discussion document for consultation with EC Fisheries Act Network
  • Finalize review document
  • EC Senior Management review of findings and implications for departmental FA priorities and work plans

Assessment 2011 :
Level 1: No progress or insignificant progress
Level 2: Planning stage
Level 3: Preparations for implementation
Level 4: Substantial implementation
Level 5: Full implementation Obsolete

PRSD/ESBEnforcement Branch, EPOD(Compro and Regions); Sector risk management groups within ESB; S&T Branch20102012
1.120EC should review existing Fisheries Act regulations, guidelines and best management practices to ensure that they are adequate, up to date, relevant and enforceable.

Environment Canada accepts this recommendation. Over 2009-2012, Environment Canada will undertake a review of the continued relevance of the four regulations noted below in light of Fisheries Act guidelines, provincial standards and industry best management practices and will take the necessary steps to update or repeal them as appropriate.

  • Chlor-Alkali Mercury Liquid Effluent Regulations
  • Meat and Poultry Product Plant Effluent Regulation
  • Petroleum Refinery Liquid Effluent Regulations
  • Potato Processing Plant Liquid Effluent Regulations

Completed

  • Identify sector leads for each regulation and guideline in June 2009
  • Preliminary assessment of Petroleum and Refinery Liquid Effluent Regulations (PRLER) was completed in 2010. It included background study on the status of refinery effluent systems in Canada, regulatory regimes, technologies for the management of liquid effluents, refinery effluent data and started analysis.

Ongoing

  • For PRLER, development of options and recommendations has not been completed due to lack of FTE funding.
  • Preliminary assessment of the Meat and Poultry Products Plant Effluent Regulations and Guidelines and the Potato Processing Plant Liquid Effluent Regulation and Guidelines initiated in 2010 with review of past assessments and analysis of information needs.

Upcoming

  • Preliminary assessment for the Chlor-Alkali Mercury Liquid Effluent Regulations not initiated; however as the mercury cell Chlor – Alkali industry no longer operates in Canada there may be no need for further assessment or consideration of this regulation.
  • Preliminary assessment of the Guidelines for Metal Finishing Liquid Effluent and Fish Processing Operations Liquid Effluent
  • Recommendations sent to EC Senior Management for decision

Assessment 2011:
Level 1: No progress or insignificant progress
Level 2: Planning stage
Level 3: Preparations for implementation
Level 4: Substantial implementation
Level 5: Full implementation Obsolete

PRSD/ESBEnforcement Branch, EPOD(Compro and Regions); Sector risk management groups within ESB20092012
1.126EC should ensure that its enforcement quality assurance and control processes are sufficient to demonstrate that its actions have been taken in accordance with the Compliance and Enforcement PolicyEnvironment Canada accepts this recommendation. The Enforcement Branch is continuing to develop a framework, standardize processes and establish accountabilities to enhance its quality assurance and its quality control. More specifically, the quality assurance and quality control framework is being both developed and implemented over the 2009-10 and 2010-2011 fiscal years and maintained thereafter. At the same time, the Enforcement Branch is establishing a quality assurance unit as well as a working group to oversee and support the quality of enforcement data. Collectively their responsibilities will include the development of new procedures for data entry, implementing a systematic data quality and control monitoring process that will involve both regional management teams as well as headquarters, conducting periodic quality assurance analysis of enforcement files and the provision of training to Enforcement Officers.

Completed

  • The Enforcement Branch’s Environmental Enforcement Directorate has established a dedicated Quality Assurance officer at headquarters and a national working group to review data quality issues on an ongoing basis.
  • The Branch produces and distributes exception reports on a quarterly basis, which are sent to regions to resolve any anomalies.
  • The Branch established National Data Input Standards in April 2009 to ensure consistent, high quality data across the country.

Assessment 2011:
Level 1: No progress or insignificant progress
Level 2: Planning stage
Level 3: Preparations for implementation
Level 4: Substantial implementation
Level 5: Full implementation Obsolete

Environmental Enforcement Directorate (EED)/ Enforcement Branch (EB) 20092011
1.134We have recommended that Fisheries and Oceans Canada conduct a review of the 1986 Habitat Policy. As part of that review, DFO should, with the support of EC, clearly establish the expectations for EC’s administration of the pollution prevention provision, including the expected interactions between the two departments to support the delivery of the Habitat Policy.Fisheries and Oceans Canada and Environment Canada accept this recommendation and Fisheries and Oceans Canada will, by March 31, 2011, complete a review of the administration of Section 36 of the Fisheries Act with Environment Canada. By March 31, 2012, a renewed Memorandum of Understanding that better establishes expectations and responsibilities for Environment Canada will be in place.

Completed

  • Initiated discussion at the 2009 workshop, followed up with a first meeting in January 2010, a second one in June 2010 and monthly calls up to November. Of note, Major Projects Management Office discussions and FA Renewal do have major impacts on this activity.
  • At the October 2010 workshop, a special session between key players took place to further advance the content of the MOU.

Upcoming

  • Internal preparation to engage in discussions with DFO towards renegotiating the MOU and resolving the issue by March 2012
  • DFO and EC will review roles and responsibilities of the respective departments in the administration of Section 36 of the Fisheries Act.
  • Propose revised MOU to senior management
  • EC will review the Compliance and Enforcement Policy (2001) in consultation with DFO and update this policy by 2012.

Assessment 2011:
Level 1: No progress or insignificant progress
Level 2: Planning stage
Level 3: Preparations for implementation
Level 4: Substantial implementation
Level 5: Full implementation Obsolete

PRSD/ESB(overall lead for MOU) and EED/EBLRAD, Legal Services20092012

[1] See logic model for a list of WRM outcomes.

[2] Formerly known as the Chief Information Officer Branch (CIOB).

[3] While primary responsibility for municipal solid waste rests with municipalities (collection, diversion and disposal) and the provinces (approval, licensing, and monitoring of operations), the federal government remains connected with the management of municipal solid waste through engagement with “issues related to sustainable development, toxic substances, international movement, federal lands and operations, air emissions, including greenhouse gas emissions, and through federal funding programs.” (EC. Municipal Solid Waste. Accessed from http://www.ec.gc.ca/gdd-mw/default.asp?lang=En&n=EF0FC6A9-1, 2010).

[4] EC. “2010-2011 Program Activity Architecture.” Internal document, 2010.

[5] Although this change officially occurred in April 2011, divisions were already organized in this manner when data were being collected for the evaluation in early 2011.

[6] Source: data supplied by Finance Branch (in April and May 2011), showing actual expenditures in the fiscal years shown for fiscal codes linked to the WRM SSA.

[7] Total financial amounts represent budgeted amounts from ESB, as well as budgeted amounts related to supporting services provided by CIOB and S&T, related to WRM activities. Enforcement and compliance promotion amounts are not available at the level of individual regulations or programs (including PAA elements).

[8] Budgeted ESB amounts are only available at the level of the PAA element (or sub-sub-activity level). Therefore, amounts are presented for the entire PAA element and not by Division.

[9] Source: data supplied by Finance Branch (in April and May 2011), showing actual expenditures in the fiscal years shown for fiscal codes linked to the WRM SSA.

[10] Based on changes to the financial coding structure in 2010-2011, whereby regional codes were deactivated and data were not able to be broken down to the same level as in previous years; therefore totals represent the sum of divisional and regional funding. Regional funding is managed through the Environmental Protection Operations Directorate (EPOD).

[11] Note that Environmental Effects Monitoring is managed by FAAD and delivered within the regions through EPOD.

[12] Note that APSD was previously within this SSA; however, this division was dissolved in April 2011, and responsibilities for wastewater were reassigned to the SWMD, while the Federal House moved to WRMD.

[13] Total financial amounts represent actual expenditures from ESB, as well as expenditures related to supporting services provided by CIOB and S&T, related to WRM activities. Enforcement and compliance promotion amounts are not available at the level of individual regulations or programs (including PAA elements).

[14] “Of the six FA pollution prevention regulations currently in force, EC actively administers two–the Pulp and Paper Effluent Regulations and the Metal Mining Effluent Regulations. The four remaining regulations date back to the 1970s and are based on outdated technology and practices, making them difficult to enforce.” (CESD. “Chapter 1: Protecting Fish Habitat.” Report of the Commissioner of the Environment and Sustainable Development. Accessed from http://www.oag-bvg.gc.ca/internet/English/parl_cesd_200905_01_e_32511.html, spring 2009.)

[15] These are the Pulp and Paper Mill Defoamer and Wood Chip Regulations and the Pulp and Paper Mill Effluent Chlorinated Dioxins and Furans Regulations.

[16] CESD. “Chapter 1: Protecting Fish Habitat.” Report of the Commissioner of the Environment and Sustainable Development. Accessed from http://www.oag-bvg.gc.ca/internet/English/parl_cesd_200905_01_e_32511.html, spring 2009.

[17] DFO. Annual Report April 1, 2007 to March 31, 2008: Annual Report on the Administration and Enforcement of the Fish Habitat Protection and PPP of the FA. Accessed from http://dfo-mpo.gc.ca/habitat/role/141/reports-rapports/2007-2008/index-eng.asp, 2008.

[18] Items within this list are drawn from EC. “2009-10 Mining Sector Workplan (conventional mining).” Internal document, n.d.

[19] A regulatory amendment to add a new TIA to Schedule 2 of the MMER triggers a federal environmental assessment. DFO is the Responsible Authority and leads the assessment. EC and other departments, depending on the mining proposal, provide technical advice and input to the assessment. Following the environmental assessment, and depending on its outcome, MPD prepares a regulatory amendment package. A new listing on Schedule 2 requires a regulatory amendment, which takes 8-12 months from the date of the environmental assessment decision.

[20] Although EC administers the MMER, the Minister of DFO retains the final approval to submit a regulatory amendment package, prepared by MPD, to Treasury Board for its consideration and approval. EC. “Current Regulatory Initiatives – PRSD.” Internal document, 2010; Government of Canada. “Regulations Amending the Metal Mining Effluent Regulations.” Canada Gazette. Part II. Vol. 140, No. 21, 18 October 2010.

[21] DFO. Annual Report April 1, 2008 to March 31, 2009, p. 27.

[22] Government of Canada. “Notice Requiring the Preparation and Implementation of Pollution Prevention Plans in Respect of Inorganic Chloramines and Chlorinated Wastewater Effluents.” Canada Gazette. Part I. Vol. 138, No. 49, 4 December 2004.

[23] EC. “Business Plan 2007-2012: Waste Reduction and Management Division.” Draft internal document, May 2007.

[24] EC. “Discussion Document on Addressing Environmental Management Gaps in the Federal House.” Draft internal document, 2010.

[25] EC. Discussion Document on Environmental Management Gaps in the Federal House. 2010.

[26] Approximately 15 000 storage tank systems exist in Canada, including 4300 fuel tanks on reserve, 475 of which are associated with schools, water and wastewater treatment facilities and diesel-fuelled electrical power generating plants. (EC. “Communications Plan - Federal Storage Tank Systems Regulations: Compliance Deadline for Identification for Existing Storage Tank Systems.” Internal document, March 2010.)

[27] EC. 2010. EC’s Risk-Based Audit Plan for 2010-2013 and Risk-Based Evaluation Plan for 2010-2015. Accessed from http://www.ec.gc.ca/doc/ae-ve/2010-2011/1280/ec-rbaep-toc.htm, 2010, p. 71.

[28] EC. “Draft Performance Measurement and Evaluation Plan – Wastewater Systems Effluent Regulations.” Draft internal document, 2010, p. 13.

[29] CESD. “Chapter 1: Protecting Fish Habitat.” Report of the Commissioner of the Environment and Sustainable Development.Accessed from http://www.oag-bvg.gc.ca/internet/English/parl_cesd_200905_01_e_32511.html, Spring 2009; CESD. “Chapter 2: Risks of Toxic Substances.” Report of the Commissioner of the Environment and Sustainable Development. Accessed from http://www.oag-bvg.gc.ca/internet/English/parl_cesd_200911_02_e_33197.html, Fall 2009.

[30] OAG. “Chapter 6: Land Management and Environmental Protection on Reserves.” Fall 2009 Report of the Auditor General of Canada. Accessed from http://www.oag-bvg.gc.ca/internet/English/parl_oag_200911_06_e_33207.html, 2009.

[31] Stratos Inc., Marbek Resource Consultants Ltd., and Kilgour and Associates Ltd. “Implementation Review of the Environmental Effects Monitoring Program.” Internal document prepared for EC, 2009.

[32] EC. Evaluation of Environment Canada’s Aboriginal Consultations on Wastewater. Accessed from http://ec.gc.ca/doc/ae-ve/2008-09/906/906_eng.html, 2009; EC. Evaluation of the National Agri-Environmental Standards Initiative. Accessed from http://www.ec.gc.ca/doc/ae-ve/2008-09/979/index_e.htm, 2009; EC. Evaluation of the Enforcement Program. Accessed from http://ec.gc.ca/doc/ae-ve/2009-2010/1076/toc_eng.htm, 2009; EC. Evaluation of Enhanced Security for the Transboundary Movement of Hazardous Waste: Lessons Learned. Accessed from http://ec.gc.ca/doc/ae-ve/2008-09/602/tdm-toc_eng.htm, 2008; and EC. Evaluation of Environment Canada's Bilateral Cooperation Program under the Multilateral Fund of the Montreal Protocol. Accessed from http://www.ec.gc.ca/doc/ae-ve/BDP-PCB-eval/toc-tdm_eng.htm, 2007.

[33] A complete list of evaluation issues and questions can be found in Annex 1. The Technical Appendices to this evaluation report contain further details regarding the methodologies used, including data collection instruments and a full list of documents consulted.

[34] The request was made on February 16, 2011.

[35] NEMISIS refers to the National Emergencies and Enforcement Information System and Intelligence System. This is the EC Enforcement Branch’s database for tracking enforcement activities.

[36] Other federal contacts included EC employees who were not members of PRSD.

[37] http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=18218&section=text.

[38] EC. Municipal Solid Waste. Accessed from http://www.ec.gc.ca/gdd-mw/default.asp?lang=En&n=EF0FC6A9-1, 2010.

[39] EC and Forest Products Association of Canada. Towards a More Innovative Air Quality Management: Proposal for a Pulp and Paper Air Quality Forum. 2004; McMaster, M.E., J.L. Parrott, and L.M. Hewitt. A Decade of Research on the Environmental Effects of Pulp and Paper Mill Effluents in Canada (1992 – 2002). Accessed from http://www.ec.gc.ca/inre-nwri/default.asp?lang=en&n=DFCDAED6-1, 2004.

[40] EC. “Aquaculture Sector Overview.” Internal presentation deck, 2008.

[41] EC. “Mining in Canada.” Internal presentation deck, 2009.

[42] EC. State of Municipal Wastewater Effluents in Canada. Accessed from http://www.ec.gc.ca/Publications/default.asp?lang=En&xml=ED03E31E-8A4E-4934-96B1-0A3697410588, 2001.

[43] CCME. Review of the State of Knowledge of Municipal Effluent Science and Research – Executive Summary. 2006.

[44] EC. Hazardous Waste and Recyclable Material. Accessed from http://ec.gc.ca/gdd-mw/default.asp?lang=En&n=39D0D04A-1, 2010.

[45] EC. Mercury – Environment and Health. Accessed from http://www.ec.gc.ca/mercure-mercury/default.asp?lang=En&n=8EE3A307-1, 2010.

[46] EC. A Guide to Understanding the Canadian Environmental Protection Act, 1999. Accessed from http://www.ec.gc.ca/lcpe-cepa/default.asp?lang=En&n=E00B5BD8-1, 2004.

[47] Government of Canada. “PCB Regulations – Regulatory Impact Analysis Statement.” Canada Gazette. Vol. 140, No. 44, 4 November 2006.

[48] Treasury Board of Canada Secretariat. The Federal Contaminated Sites and Solid Waste Landfills Inventory. Accessed from http://www.tbs-sct.gc.ca/fcsi-rscf/home-accueil-eng.aspx, 2008.

[49] Federal Contaminated Sites Action Plan Secretariat. Federal Contaminated Sites Action Plan Annual Report 2007-2008. Accessed from http://www.federalcontaminatedsites.gc.ca/ar-ra/2007-2008/index-eng.aspx, 21 January 2010.

[50] Mining Association of Canada. Facts and Figures 2010: A Report on the State of the Canadian Mining Industry. 2010.

[51] CCME. Canada-wide Strategy for the Management of Municipal Wastewater Effluent. 2009.

[52] Office of the Auditor General of Canada. 2009 Fall Report of the Auditor General of Canada. “Chapter 6–Land Management and Environmental Protection on Reserves.” 2009.

[53] EC. Minister Prentice Highlights the Environment in 2010 Budget. Accessed from http://www.ec.gc.ca/default.asp?lang=En&n=FFE36B6D-1&news=BB63CA28-000A-4D63-8C15-173E257C054E, 4 March 2010.

[54] EC. Planning for a Sustainable Future: a Federal Sustainable Development Strategy for Canada. Accessed from http://www.ec.gc.ca/Publications/default.asp?lang=En&xml=722B66A7-1BD4-4287-B8AA-6B589FEFA06E, 2010.

[55] Represented in all EC Reports on Plans and Priorities from 2007-08 to present. Wording has changed slightly from year to year.

[56] EC. “FA-PPP: RMAF and Risk-Based Audit Framework (RBAF) Draft Program Profile.” Draft internal document, September 2009.

[57] EC. “Regulatory Strategy for the Movement of Wastes and Hazardous Recyclable Materials.” Internal presentation deck, 2009.

[58] It was not possible to determine whether these results reflect a lack of clarity with the WSER requirements or the fact that the regulations are still under development and as such monitoring and reporting is not yet taking place.

[59] Although those interviewed refer to the “toxicity test” broadly, issues that were identified are related to the acute lethality test.

[60] Ratings for the MMER were also lower for this outcome; however, they are based on an extremely small sample size (n=4) and as such are too small to note.

[61] See Table 10 for detailed responses.

[62] CCME. Canada-wide Strategy for the Management of Municipal Wastewater Effluent. 2009.

[63] Information taken from the following reports: EC. Summary Review of Performance of Metal Mines Subject to the MMER (2004 – 2010). Accessed from http://www.ec.gc.ca/Publications/default.asp?lang=En&xml=CD4C0FD1-A1AF-48B5-B99E-85F501FCB8CE.

[64] EC. “Status Report on Water Pollution Prevention and Control under the Pulp and Paper Effluent Regulations in 2008.” Internal document, n.d.

[65] Information obtained through the Federal Identification Registry for Storage Tank Systems (FIRSTS) and reporting in EC. “Implementation of the Storage Tank Systems for Petroleum Products and Allied Petroleum Products Regulations 2010.” Draft internal document, 2010.

[66] See Table 10 for detailed responses.

[67] EC. “Memorandum to the Assistant Deputy Minister, ESB: Approach for Minimizing the Potential of the Minister Appearing of Denying Fuel to the First Nations Communities.” Internal document, 11 January 2010.

[68] EC. “Storage Tanks Identified on Aboriginal Lands under Federal Storage Tank Regulations.” 9 December 2010.

[69] Information obtained through the Federal Identification Registry for Storage Tank Systems (FIRSTS) and reporting in EC. “Implementation of the Storage Tank Systems for Petroleum Products and Allied Petroleum Products Regulations 2010.” Draft internal document, 2010.

[70] The PCB Regulations were introduced in September 2008. The objective of these regulations, which include deadlines for the end of use and storage of PCBs in Canada and timelines for disposal, is to significantly decrease releases of PCBs into the Canadian environment (EC. “DRAFT National Enforcement and Compliance Promotion Planning Process 2011-2014 – PCB Regulations.” Draft internal document, n.d.; EC. “National Enforcement Plan 2010-11.” Internal document, n.d.)

[71] This compliance result is based on compliance promotion and discussions with regulatees on a voluntary return to compliance, which preceded any enforcement actions. Compliance promotion data are currently qualitative in nature and rely on data provided by Enforcement through NEMISIS as well as compliance promotion activity. A new, more comprehensive database of compliance promotion data is currently under development.

[72] EC. “DRAFT National Enforcement and Compliance Promotion Planning Process 2011-2014 – PCB Regulations.” Draft internal document, n.d.

[73] EC. Evaluation of the Enforcement Program. Accessed from http://ec.gc.ca/doc/ae-ve/2009-2010/1076/toc_eng.htm, 2009.

[74] The compliance results are based on a weighted average of compliance results provided for both on-site and off-site inspections as reported by the Enforcement Branch in NEMISIS. Compliance results for 2010-11 are based on compliance data provided up to June 2011.

[75] Represents the first year of measuring compliance against the 2008 “end-of-use” requirement.

[76] This outcome pertains specifically to WRMD.

[77] See Table 10 for detailed responses.

[78] Statistics on the International Movements of Hazardous Waste and Hazardous Recyclable Material, 2007-9.

[79] This outcome pertains specifically to WRMD.

[80] EC, International Affairs Branch. Compendium of International Environmental Agreements, September 2008, 4th edition.

[81] EC. EIHWHRMR User Guide. http://www.ec.gc.ca/gdd-mw/default.asp?lang=En&n=5D5BE79E-1.

[82] EC. Compendium of International Environmental Agreements. September 2008, 4th edition.

[83] EC. “Performance Measurement and Evaluation Plan – Products Containing Certain Toxic Substances Regulations.” Internal document, 2010.

[84] Canada: National Reporting to Comission on Sustainable Development-18/19 Thematic Profile: Waste Management. Accessed from www.un.org/esa/dsd/dsd_aofw_ni/ni.../canada/Waste_Management.pdf.

[85] See Table 10 for detailed responses.

[86] See Table 10 for detailed responses.

[87] Information taken from the following reports: EC. Summary Review of Performance of Metal Mines Subject to the MMER (2004 – 2010).

[88] EC. “Status Report on Water Pollution Prevention and Control under the Pulp and Paper Effluent Regulations in 2008.” Internal document, n.d.

[89] Information obtained through the Federal Identification Registry for Storage Tank Systems (FIRSTS) and reporting in EC. “Implementation of the Storage Tank Systems for Petroleum Products and Allied Petroleum Products Regulations 2010.” Draft internal document, 2010.

[90] See Table 10 for detailed responses.

[91] This outcome pertains specifically to WRMD.

[92] See Table 10 for detailed responses.

[93] OAG. “Chapter 6: Land Management and Environmental Protection on Reserves.” Fall 2009 Report of the Auditor General of Canada. Accessed from http://www.oag-bvg.gc.ca/internet/English/parl_oag_200911_06_e_33207.html, 2009.

[94] See Table 10 for detailed responses.

[95] “don’t know” responses were removed from the analysis for ease of comparison.

[96] EC. “2009-10 Planning and Resource Requirement (OPG-3A4): Risk Management – 3A4. EP Board.” Internal presentation deck, n.d., p. 8.

[97] During the evaluation the branch was reorganized and now falls within the Corporate Services Branch.

[98] EC. “Draft ECFA-PPP Logic Model.” Draft internal document, December 2010.

[99] EC. “3A4C: Waste Management and Reduction Program – Risk Management Instruments.” Draft internal document, 7 May 2010.

[100] EC. “Performance Measurement Plan for the Storage Tank Systems for Petroleum Products and Allied Petroleum Products Regulations.” Draft internal document, 11 October 2009; EC. “Storage Tank Program Logic Model.” Draft internal document, 26 July 2010.

[101] EC. “Draft Performance Measurement and Evaluation Plan – WSER.” Draft internal document, 2010.

[102] EC. “Implications of the Lack of Critical Upgrades to Search and Reporting Functions for the FIRSTS Database.” Internal document, 2010.

[103] Government of Canada, June 6, 2011, The Next Phase of Canada’s Economic Action Plan. A Low Tax Plan for Jobs and Growth. (Budget 2011-12).

[104] EC. “2009-10 Planning + Resource Requirement (OPG-3A4): Risk Management – 3A4. EP Board.” Internal presentation deck, n.d., p. 8.

[105] Mazur, E. “Outcome Performance Measures of Environmental Compliance Assurance: Current Practices, Constraints and Ways Forward.” OECD Environment Working Papers, No. 18. OECD Publishing, 2010.

[106] Stratos Inc. “Implementation Review of the EEM Program.” Internal document prepared for EC, February 2009.

[107] The following articles were examined: Kagan, Robert, Dorothy Thornton and Neil Gunningham. “Explaining Corporate Environmental Performance: How Does Regulation Matter?” Law & Society Review. Vol. 37, No. 1, 2003, pp. 51-90; Harrison, Kathryn and Werner Antweiler. “Incentives for Pollution Abatement: Regulation, Regulatory Threats, and Non-Governmental Pressures.” Journal of Policy Analysis & Management. Vol. 22, No. 3, Summer 2003, pp. 361-382; Domfeh, Kwame Ameyaw. “Compliance and Enforcement in Environmental Management: A Case of Mining in Ghana.” Environmental Practice. Vol. 5, No. 2, 2003, pp. 154-165; Dashwood, Hevina. “Canadian Mining Companies and Corporate Social Responsibility: Weighing the Impact of Global Norms.” Canadian Journal of Political Science,Vol. 40, No. 1, 2007, pp. 129-156; and Keremane, Ganesh and Jennifer McKay. “Successful wastewater reuse scheme and sustainable development: a case study in Adelaide.” Water and Environment Journal, Vol. 21, No. 2, 2007, pp. 83–91.

[108] EC. “Performance Measurement and Evaluation Plan – Products Containing Certain Toxic Substances Regulations.” Internal document, 2010.

[109] EC. “Proposed Regulations on Storage Tank Systems on Federal and Aboriginal Land.” Internal document, 27 November 2002.

[110] Due to EC’s resource reallocation processes it is unclear whether this money was available to the SSA at year end.

[111] Discussions with Finance revealed that although O&M expenditures related to compliance promotion could be coded to each regulation through the use of project activity codes, in practice, these codes are not consistently applied, and therefore actual expenditures related to individual regulations are unreliable.

[112] EC. Wastewater Systems Effluent Regulations: Regulatory Impact Analysis Statement. March 2010. Accessed at http://www.gazette.gc.ca/rp-pr/p1/2010/2010-03-20/html/reg1-eng.html.

[113] A complete description of the recommendations, management response, and 2011 update can be found in Annex 3.

[114] Formally known as the Chief Information Officer Branch (CIOB).

[115] A complete description of the recommendations, management response, and 2011 update can be found in Annex 3.

[116] In order to be counted toward a department’s evaluation coverage of 100% of direct program spending every five years, evaluations of federal programs, policies and initiatives are required to assess all core issues identified by the Treasury Board Directive on the Evaluation Function (http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15681). This column identifies which of the core evaluation issues outlined in the Treasury Board directive is addressed by each evaluation question.

[117] The rating symbols and their significance are outlined in Table 8 of the report.

Date modified: