Environment Canada's Three-Year Risk-Based Audit and Evaluation Plan 2009-2012

Back

6. Evaluation Plan – 2009-2010 to 2011-2012

6.2 2010-2011
No. Project Name Board Alignment
to Strategic Outcomes
Rationale Scope & Objectives Planned Required Resources Risk Rating Priority Rating
PM $’000 L/M/H L/M/H *man-
dated
34. Adaptation Thematic Roll-up EP Canadians and their environment are protected from the effects of pollution and waste This evaluation responds to a Treasury Board Secretariat requirement. The objective of this project is to roll-up in a thematic evaluation the findings of various individual Adaptation Theme evaluations, including evaluations of the National Air Quality Health Index and Climate Change Scenarios led by EC in 2009-2010.

The results of the evaluation will inform future adaptation initiatives and feed into a CAA level roll-up of evaluation findings from across the eight theme areas.
    M M*
35. Clean Air Agenda (CAA) Horizontal Roll-up EP Canadians and their environment are protected from the effects of pollution and waste Through the Clean Air Agenda (CAA), the Government of Canada is working towards meeting its commitments to tangible improvements in Canada's environment, including reduced air pollution and greenhouse gas (GHG) emissions. The CAA is a four-year, $1.9-billion horizontal strategy involving the Clean Air Regulatory Agenda ($0.35 billion) and complementary program measures ($1.55 billion). Eight federal departments and agencies are involved in designing and delivering more than 40 programs under 8 themes within the overall Agenda.

The Clean Air Agenda encompasses a multitude of programs and initiatives organized under several themes. Each theme is required to be evaluated through either a theme-level evaluation or roll-up of theme program evaluations prior to the completion of the four-year funding period. The results of the seven theme evaluations will then be rolled-up into a single umbrella report to provide an overall assessment of the CAA.

This evaluation responds to a Treasury Board Secretariat requirement.
The objective of this project is to roll-up the results of the eight theme evaluations into a single umbrella report to provide an overall assessment of the Clean Air Agenda.     M M*
36. Federal Species at Risk Program

(Interdepartmental led by EC)
ES Canada’s natural capital is restored, conserved, and enhanced In 2006, a formative evaluation of the SAR Program was completed; while this evaluation demonstrated important progress, it did identify significant outstanding challenges. A management action plan to respond to the evaluation’s recommendations was developed and implementation is closely tracked.

The Federal Species at Risk Program has recently undergone a reorganization to simplify the Program structure by merging the Sub sub-activities. Nonetheless, the Program can be characterized by a high degree of complexity given its decentralized operations and the processes associated with COSEWIC. The Program also has a high degree of compliance requirements and obligations, in particular the Species at Risk Act, Canadian Biodiversity Strategy, Accord for the Protection of Species at Risk in Canada, and various Bilateral agreements under SARA.

There is a high level of knowledge required to properly conduct operations for this Program and it is highly dependent on external parties for successful operations (for example cooperative arrangements with provinces/territories, provision of funding to Aboriginal people and stakeholders for implementation). Environment Canada is exposed to risks associated with failures on the part of third parties on which it relies to deliver the program.

This Program is also inherently exposed to risks associated with natural hazards.

This evaluation responds to a Treasury Board Secretariat requirement.
The objective of this project is to conduct a summative evaluation of the Federal Species at Risk Program (including the Aboriginal Funds Species at Risk [AFSAR] Program).     M M*
37. Evaluation of the Lake Simcoe Freshwater Initiative ES Canada’s natural capital is restored, conserved, and enhanced Environment Canada is carrying out its work under the Action Plan for Clean Water through $96 million in clean-up funding to restore Lake Simcoe, Lake Winnipeg, and Areas of Concern in the Great Lakes.

The CESD audited Water in 2001 and Drinking Water in 2005 and encouraged Environment Canada to reassess its role and develop clear action plans to carry out its commitments for management of freshwater water.

This evaluation responds to a Treasury Board Secretariat requirement.
The objective of this project is to evaluate the Lake Simcoe program as part of the Action Plan for Clean Water.     M M*
38. Evaluation of the Lake Winnipeg Freshwater Initiative ES Canada’s natural capital is restored, conserved, and enhanced Environment Canada is carrying out its work under the Action Plan for Clean Water through $96 million in clean-up funding to restore Lake Simcoe, Lake Winnipeg, and Areas of Concern in the Great Lakes.

The CESD audited Water in 2001 and Drinking Water in 2005 and encouraged Environment Canada to reassess its role and develop clear action plans to carry out its commitments for management of freshwater water.

This evaluation responds to a Treasury Board Secretariat requirement.
The objective of this project is to evaluate the Lake Winnipeg program as part of the Action Plan for Clean Water.     M M*
39. Northwest Territories Protected Areas Strategy Program ES Canada’s natural capital is restored, conserved, and enhanced Environment Canada is currently developing a Protected Areas Strategy. The Northwest Territories Protected Areas Strategy Program is characterized by a high degree of complexity, for example operations are decentralized and activities must be planned, coordinated and managed across the region and nationally. This exposes Environment Canada to risks associated with a lack of alignment between key resources and program operational priorities, which could compromise the delivery of the full scope of the mandate. The Protected Areas Management Program is struggling to manage protected sites effectively within the current budget.

The Program is also subject to a high degree of compliance requirements and obligations, specifically the role derived from Canada Wildlife Act, the Migratory Birds Convention Act, the Species at Risk Act, and the Inuit Impact and Benefit Agreement. This exposes the Department to risks associated with the violation of laws, regulations, international treaties/agreements and policies.

The significant number of dependencies in the Program also exposes the Department to risks associated with failures on the part of third parties on which Environment Canada relies. For example, the Department depends on other government departments such as PCA, DFO and INAC, the government of Northwest Territories and Aboriginal groups on Northwest Territories Protected Areas Strategy, the government of Nunavut and Nunavut Tunngavik Incorporated, and non-governmental organizations.

This evaluation also responds to a Treasury Board Secretariat requirement.
The objective of this project is to conduct a formative evaluation of the NWT Protected Areas Strategy Program. The evaluation will assess the extent to which the program is meeting its outcomes/ results and identify areas where adjustments could be made.     H H*
40. Policy on Green Procurement

(Interdepartmental led by PWGSC)
IS Integration and Enabling Services contribute to achieving departmental strategic objectives As indicated in the 2008-2009 Report on Plans and Priorities, DMS priorities include work to support the greening of federal government operations. Specific priorities include the integration of environmental performance drivers, especially with respect to the management of Environment Canada’s contaminated sites, green procurement, vehicle fleet, greening of IT assets, and reduction of GHG emissions. In addition, it is a priority to develop new approaches to assets management and capital planning in line with government-wide directions and greening government initiatives.

Environment Canada is one of the co-champions for the Government of Canada regarding the greening of operations. The CESD follow-up to Management Tools and Government Commitments — Greening of Government Operations reported that the government's progress toward providing departments preparing sustainable development strategies with guidance on greening their operations was unsatisfactory.

This evaluation responds to a Treasury Board Secretariat requirement.
The objective of this project is to evaluate the Policy on Green Procurement. The focus of the evaluation will be on the impact and net effect of the Policy.     M M*
41. Federal Contaminated Sites Action Plan EP Canadians and their environment are protected from the effects of pollution and waste $80.5 million will be available over the next two years for the management and assessment of federal contaminated sites.

FCSAP is a highly complex horizontal initiative that is being delivered by fifteen federal custodians. Environment Canada co-leads administration of this program with the Treasury Board Secretariat and is governed by an ADM-level Steering Committee. As such, this initiative exposes Environment Canada to risks associated with both business processes due to its complexity and third party dependency, and in particular a dependency on federal financial assistance programs that provide funding for activities related to federal contaminated sites.

This initiative also requires a high degree of knowledge; this exposes Environment Canada to risks associated with knowledge capital.

In addition to the above rationale, this evaluation will respond to a Treasury Board Secretariat requirement.
The objective of this project is to conduct a summative evaluation of the Federal Contaminated Sites Action Plan.

The timing of this project is still to be determined.
    H H*
42. Regulatory Improvements for Major Resource Projects

(Interdepartmental, led by Natural Resources Canada)
EP Canadians and their environment are protected from the effects of pollution and waste This evaluation responds to a Treasury Board Secretariat requirement that NRCan returns to Cabinet before March 31, 2012 to report on progress; as such, an evaluation is anticipated before this date. The scope and objectives will be decided when the project is confirmed.     M M*
43. Ice Program WES Weather and environmental predictions and services reduce risks and contribute to the well-being of Canadians Environment Canada puts approximately $3 million into the Ice Program and the Department of Fisheries and Oceans spends approximately $8–9 million.

The Ice Program is fairly complex given a service partnership agreement with the Canadian Coast Guard, which exposes the Department to risks associated with adequate business processes and third party dependency; this is compounded by the Program’s reliance on infrastructure support from the Chief Information Operations Branch.

This Program has not been previously evaluated, though a similar project was approved in the 2007-2008 plan – it was postponed from 2008-2009 to 2009-2010 due to work capacity issues brought about by the Strategic Review exercise. A decision was made by management to postpone again from 2009-2010 to 2010-2011 to accommodate other high priority projects.
The objective of this project is to evaluate a recent application of changes in technology and program delivery in terms of safeguarding public safety.     L L
44. Stakeholders and Partnership Management (joint audit-evaluation – collaborative approach) IS Integration and Enabling Services contribute to achieving departmental strategic objectives Environment Canada has a high dependence on partners and stakeholders for program delivery and financial support. Lack of formal agreements can lead to further erosion of public confidence in the Department’s information and services. As such, Stakeholder and Partnership Management was identified as a risk in the 2008 Corporate Risk Profile (legal, financial, loss of programs/services).

Stakeholder and partnership management was also identified as a risk in 2008-2009 and 2009-2010 AEB risk assessments, which highlighted the complexity of partnership arrangements and agreements. There is a risk of fragmented efforts and decreased cooperation and consensus. Performance in this area has been affected by a recent departmental transformation. Specific risks are associated with G&C process and it is recognized that there are inconsistencies in departmental approach.

No audits or evaluations have been conducted in this area.
The objective of this project is to evaluate stakeholder and partnership management across the Department.

This is a joint audit and evaluation project.

The evaluation will address bilateral (e.g. academic, scientific) and federal-provincial relationships/agreements.
    H H
45. Water Management (joint audit-evaluation – collaborative approach) ES Canada’s natural capital is restored, conserved, and enhanced Water management was identified as a risk in the 2009-2010 AE Risk Assessment and management consultations. Water Management is a highly visible and complex program with many different activities throughout the Department that contribute to the achievement of its mandate. Because of the complexity of the program, the Department is exposed to risks associated with the misalignment of activities, priorities and financial resources and risks associated with inappropriate or inadequate business processes. The Program also requires a high level of scientific knowledge.

Water Management is subject to a moderate level of compliance, characterized by various legal commitments that include domestic Water Boards, Binational / IJC Boards, International River Improvements Act (IRIA), Canadian Water Act (CWA). This exposes the Department to risks associated with violation of laws, regulations, international treaties/agreements and policies.

The Program also has many dependencies with other programs, relies on collaboration with sub-activities, other federal departments, private and non-profit sector organizations. Due to the high degree of dependencies, there are risks associated with the failure on the part of third parties on which the Department depends. In addition, bilateral partners are developing their own water plans and asking about the federal position.

The 2005 CESD audit of Drinking Water observed that the current status of the federal water Framework was unclear and its future uncertain. In 2008, the CESD found that EC has made unsatisfactory progress on issues raised in the 2001 Water audit. While the department has recently clarified some responsibilities, it still has not clearly specified who is responsible for carrying out all the required remedial actions, who will pay for those efforts, and within what timelines the actions will be taken. In addition, no formal policy or performance framework exists for this program, though a federal Water Strategy review is currently underway.

Overall, there is a risk that Environment Canada will not be able to sustain a leadership and management capacity with the right competencies and knowledge to meet its requirements.
The objective of this project is to evaluate the appropriateness, design, effectiveness, and efficiency of Environment Canada’s current approach to water management across the Department.

This is a joint audit and evaluation project.
    H H
46. Weather Prediction Program - Dissemination Systems and Service Delivery WES Weather and environmental predictions and services reduce risk and contribute to the well-being of Canadians This area was identified as a risk in the 2009-2010 AE Risk Assessment. There is currently an imbalance between Meteorological Service of Canada (MSC) capacity to generate information and its capacity to make it accessible to Canadians. Insufficient capacity on the delivery side is resulting in a failure to realize the full potential safety and economic benefits for this information. This imbalance is negatively affecting all delivery channels. A significant number of line organizations are directly involved in delivery of the services making national consistency hard to maintain. The WeatherRadio network is not resourced to allow for full life cycle maintenance and there are insufficient resources to correct the deficiencies in the automated telephone network.

There has also been some difficulty meeting all of the requirements of the Official Languages Act. (The Commissioner of Official Languages has planned a follow-up audit on Automated Bilingual Weather and Environmental Information for Fall 2010.)
The objective if this project is to evaluate the delivery of weather and environmental information services to Canadians.

This evaluation will provide an opportunity to assess performance expectations from recent ISO certification.
    H M
47. Greening Initiatives/ Sustainability IS Integration and Enabling Services contribute to achieving departmental strategic objectives Environment Canada’s greening and sustainability initiatives were identified in the 2008-2009 and 2009-2010 AE Risk Assessments because of challenges in demonstrating how efforts contribute to the Sustainable Development Strategy, affecting the clear articulation of direction and priorities. A CESD audit on the Federal Sustainable Development strategy and selected departmental commitments is expected in November 2009.

As indicated in the 2008-2009 Report on Plans and Priorities, DMS priorities include work to support the greening of federal government operations. Specific priorities include the integration of environmental performance drivers, especially with respect to the management of Environment Canada’s contaminated sites, green procurement, vehicle fleet, greening of IT assets, and reduction of GHG emissions. In addition, it is a priority to develop new approaches to assets management and capital planning in line with government-wide directions and greening government initiatives.

Environment Canada is one of the co-champions for the greening of operations goal of the Government of Canada. The CESD follow-up to Management Tools and Government Commitments — Greening of Government Operations reported that the government's progress toward providing departments preparing sustainable development strategies with guidance on greening their operations was unsatisfactory.
The objective of this project will be to evaluate the Department’s progress regarding greening initiatives and recent sustainability policy. The evaluation will assess compliance and performance (efficiency and effectiveness) under the new Sustainable Development Strategy (SDS) and Sustainable Development Act and the performance of meeting greening government initiatives (e.g. green procurement, assets management/capital planning, and fleet management).

Note: See the same topic listed under Internal Audit Projects planned for 2010-2011. It is still to be determined whether this project will be conducted as an audit or evaluation; this is potentially a joint project.
    M M
48. Grant or Contribution (TBD) TBD TBD This evaluation responds to a Treasury Board Secretariat requirement for renewal and Federal Accountability Act requirement for Gs & Cs. The scope and objectives for this project will be determined when the project is identified.       *
49. Grant or Contribution (TBD) TBD TBD This evaluation responds to a Treasury Board Secretariat requirement for renewal and Federal Accountability Act requirement for Gs & Cs. The scope and objectives for this project will be determined when the project is identified.       *
50. Performance Information in Strategic Planning and Reporting Documents (joint audit-evaluation – collaborative approach) IS Integration and Enabling Services contribute to achieving departmental strategic objectives The recent Budget identified four environmental priorities under the general theme of “A Better Canada: Ensuring a Cleaner, Healthier Environment”( Cleaner energy and better energy efficiency; Promoting cleaner transportation; Protecting Canada’s natural heritage; Canada’s National Water Strategy). In light of this emphasis on the agenda EC is largely responsible for, meeting its reporting obligations will be particularly important for the Department.

Meeting significant expectations on performance measurement and reporting aspects of a government-wide management framework for the environmental agenda was also one of the 2008-2009 DMS Board priorities.

As well, Round V of the MAF assessment had assigned an “acceptable” rating to both the Quality of Reporting to Parliament and the Effectiveness of the Corporate Management Structure (AOM #7) at EC. Yet, the assessment had also recommended that the planning documents at the branch level at EC should be more comprehensive and have clearer links to priorities.

Risks to strategic planning and performance reporting processes have been identified in both the 2008-2009 and 2009-2010 AE Risk Assessments, underlining the importance of an audit of these areas. The risk that strategic directions and objectives are not communicated and aligned with the mandate and operational plans, and the risk associated with the changing governmental governance approach were noted in particular.

While risks related to performance reporting have been reduced due to the mapping of the result structure against the PAA, potential control issues around performance information (accuracy, relevance, usefulness) are possible, making an audit of the DPR and RPP needed.

It should be noted that integrated planning (HR, finance, admin) is a priority for the Clerk of the Privy Council and makes this a higher inherent risk for EC.

Performance information is critical data for use in evaluations, thus it is important to assess the availability and relevance of this information.
The objective of this project will be to assess the accuracy, consistency, timeliness, and relevance of performance information for corporate planning and reporting.

This project will be conducted as a joint audit-evaluation.
    M H
51. Greenhouse Gas Data Collection, Management and Reporting (joint audit-evaluation – collaborative approach) EP Canadians and their environment are protected from the effects of pollution and waste Greenhouse gas data collection, management, and reporting was identified as a risk in the 2009-2010 AE Risk Assessment and management consultations. This Program is subject to many legislative and compliance requirements with specific international reporting obligations under both the United Nations Framework Convention on Climate Change (UNFCCC) and the Kyoto Protocol. An evaluation in this area is particularly timely and relevant in the context of the upcoming Climate Conference scheduled for December 2009 in Copenhagen.

Authorities under Canadian Environmental Protection Act 1999 (CEPA) also require the collection of facility level greenhouse gas data for those facilities that emit more than 100 kilotonnes annually of carbon dioxide or equivalents. Regulatory reporting requirements for greenhouse gases under Clean Air Regulatory Agenda (CARA) are also included. As such, Environment Canada is exposed to risks associated with non-compliance.

There is also a high level of scientific knowledge required for this Program, meaning the Department is exposed to risks associated with the recruitment and retention of human resources.

Information gathered by this program is also used for OAG performance audits; a CESD audit of the Kyoto Protocol Implementation Act is planned for Spring 2011.

This evaluation replaced the evaluation of the Climate Change Inventory that was requested by management and included in the 2008-09 AEB plan for 2009-2010. Following discussions with the Departmental Evaluation Committee and the Executive Management Committee, it was decided to postpone this evaluation to 2010-2011.
The objective of this project is to evaluate the performance of greenhouse gas data collection, management and reporting.

A key aspect of this evaluation will be the assessment of performance in meeting international obligations.

This project is a joint audit-evaluation.
    H H

Back