This page has been archived on the Web
Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.
Evaluation of the Great Lakes Action Plan IV
In the 2009-2010 fiscal year, Environment Canada's Audit and Evaluation Branch, Evaluation Division, commissioned an evaluation of the Great Lakes Action Plan IV (GLAP IV). This program was selected for evaluation prior to the end of the program's terms and conditions at the close of the 2009-2010 fiscal year.
This Final Report presents the findings of the evaluation. Chapter One provides background information on GLAP IV. Chapter Two discusses the objectives of the evaluation and the methodological approach. Chapter Three presents the evaluation's findings related to relevance, design and delivery, program outcomes, cost-efficiency and cost-effectiveness. Conclusions are included in Chapter Four and recommendations in Chapter Five.
1.1 Program Profile
The five Great Lakes – Superior, Michigan, Huron, Erie and Ontario – hold one fifth of the freshwater on the Earth's surface and 80 percent of the lake and river water in North America. Approximately 30 per cent of Canadians live within the Great Lakes Basin and 45 per cent of Canada's industry is located in this area. Ensuring environmental quality in this important region has implications for the natural environment, the health and well-being of Canadians, and Canada's economic competitiveness.
The overarching mechanism for protecting and restoring the Great Lakes is the Great Lakes Water Quality Agreement (GLWQA) (1972, 1978, 1987), which is a binational agreement between Canada and the United States. The Agreement sets out a series of commitments for both countries with respect to the Great Lakes. The International Joint Commission(IJC) assists the parties (Canada and the United States) in the implementation of the GLWQA. The most recent amendment to the Agreement was in 1987 and negotiations are under way for a renewed GLWQA. A key feature of the 1987 Agreement was the requirement that Canada and the United States take remedial action in heavily degraded locations or "areas of concern" (AOCs) around the lakes. AOCs were identified in 1985 by the IJC and relevant federal, provincial and state governments. In total, 17 AOCs were identified in Canada – twelve in Ontario and another five (along connecting channels) shared by Canada and the United States.
Canada's responsibility for managing and protecting the Great Lakes involves both federal and provincial (Ontario) jurisdictions. Both levels of government share authority to protect the environment, and are involved in aspects such as water, agriculture, species and spaces, and fisheries. Coordination of federal and provincial efforts with respect to the Great Lakes is accomplished through the Canada Ontario Agreement Respecting the Great Lakes Basin Ecosystem (COA). The goals and objectives of the COA parallel those of the GLWQA, and thus the Agreement includes an Annex on AOCs.
At the federal level, the Great Lakes Basin Ecosystem Initiative (GLBEI) is Environment Canada's mechanism for coordinating and delivering on federal commitments stemming from the GLWQA and the COA with respect to the Great Lakes Basin ecosystem. Remediation of domestic and binational Great Lakes AOCs is one of three priorities of the GLBEI.
The Great Lakes Action Plan for Areas of Concern is the primary vehicle under the GLBEI for the federal government, with Environment Canada as the lead department, to act to restore AOCs around the Great Lakes area and fulfill Canadian commitments under the GLWQA. The current fourth phase of the Action Plan, GLAP IV, was approved in 2005 with a budget of $40 million ($8 million per year over five years) to restore, protect and conserve AOCs around the Great Lakes.
1.1.1 Program Goals
The GLWQA states that "...the Parties, in cooperation with the State and Provincial Governments and the Commission, shall identify and work toward the elimination of ... Areas of Concern... Remedial Action Plans ... shall embody a systematic and comprehensive ecosystem approach to restoring and protecting beneficial uses in Areas of Concern... The Parties shall cooperate with State and Provincial Governments to ensure that Remedial Action Plans are developed and implemented for Areas of Concern."
The goals of GLAP IV are:
- to make progress on federal actions in eight AOCs: St. Mary's River, St. Clair River, Toronto and Region, Hamilton Harbour, Jackfish Bay, Detroit River, Niagara River and Port Hope; and
- to complete federal actions in seven AOCs: Thunder Bay, Nipigon Bay, Peninsula Harbour, Bay of Quinte, Wheatley Harbour, St. Lawrence River and Spanish River.
To achieve these goals, GLAP IV is based on an ecosystem approach: remedial efforts targeted to AOCs under GLAP IV deal with interrelated environmental and sustainable development issues. As laid out in the GLWQA, remediation in each AOC is guided by Remedial Action Plans (RAPs). The original RAP reports were developed in 1987 by the federal and Ontario governments, with extensive public consultation. RAPteams with scientific/technical expertise for each AOC identified the nature and causes of environmental problems, and recommended actions, implementation plans and monitoring strategies. RAP reports are issued and updated for three stages of implementation: Stage 1 – Problem definition; Stage 2 – Selection of remedial measures; and Stage 3 – Restoration of beneficial uses.
The definition and characterization of the environmental problems in the AOCs is founded on identification of the beneficial uses that are impaired, the degree of impairment and the geographic extent of such impairment. Beneficial use is described as "the ability of living organisms to use the ecosystem without adverse consequence." There are 14 beneficial use impairments (BUIs) that are assessed:
- Restrictions on fish consumption
- Tainting of fish and wildlife flavour
- Degradation of fish and wildlife populations
- Fish tumours or other deformities
- Bird (or other animal) deformities or reproduction problems
- Degradation of benthos (organisms living in or near marine sediment environments)
- Restrictions on dredging activities
- Eutrophication or undesirable algae
- Restrictions on drinking water consumption, or taste and odour problems
- Beach closings / water contact‑sports restrictions
- Degradation of aesthetics
- Added costs to agriculture or industry
- Degradation of phytoplankton and zooplankton populations
- Loss of fish and wildlife habitat
Prior to the renewal of the GLAP in 2005, a total of 98 BUIs were identified across the 15 Canadian AOCs, and 14 additional beneficial uses required further evaluation. Using this information, RAP reports were updated and the priority work required in each AOC for delisting was identified. Removal of the designation as an AOC and monitoring of recovery indicates that the identified beneficial uses are no longer impaired and the area is restored. An Area in Recovery is a designation used to indicate that all remedial actions have been taken and the area must enter a period of natural recovery.
1.1.2 Program Activities
Upon renewal of the GLAP in 2005, Environment Canada, with federal partners, undertook a work planning and priority-setting effort. The federal partners that participated were the National Water Research Institute (NWRI) of Environment Canada, the former Environmental Conservation Branch (ECB), the Department of Fisheries and Oceans (DFO), and Public Works and Government Services Canada (PWGSC).
The recommendations of the GLAP Work Plan Review Team were approved by the federal Great Lakes Executive Committee (GLEC) and signed off by the Environment Canada GLAP program director. GLAP IV funding was distributed to Environment Canada, DFO and PWGSC in April 2006.
Three broad categories of priority investments under GLAP IV within AOCs are:
- Remedial actions – Remedial actions within AOCs, consistent with the mandate of the Department, typically administered through the Great Lakes Sustainability Fund (GLSF):
- contaminated sediment assessment activities, such as decisions on sediment management measures, and completion of contaminated sediment risk management strategies or engineering studies;
- habitat restoration activities, such as supporting habitat creation projects, supporting stewardship and habitat acquisition, and habitat identification, restoration and protection;
- support to municipal infrastructure improvements, including pre-engineering design studies for infrastructure upgrades, actions to secure partner funding and municipal commitment, and evaluation of cost-effective technologies for municipal wastewater treatment.
- Science – Assessment and monitoring of BUIs:
- assessments of ecosystem health based on the 14 BUIs identified in the GLWQA, to determine the need and effectiveness of remedial actions and eventually to confirm restoration of ecosystem health and therefore support official delisting of the AOC.
- Engagement/Governance – Cooperative development and updating of RAPs with partners, consultation with the public:
- management of federal programs in AOCs and coordination with Ontario and local governments, the private sector, non-governmental organizations (NGOs) and individuals engaged in the remediation of AOCs.
Excluding GLSF projects, approximately 40 federal science and monitoring projects were recommended for implementation over the life of the program. Table 1.1 displays these projects by AOC priority grouping and Table 1.2 by investment priority. This information is drawn from the recommended work plans for federal partners; information on actual project expenditures is not known, due to gaps in financial tracking of this information within Environment Canada (discussed in section 3.2.4).
|Group 1 Priority AOCs||Group 2 Priority AOCs||Total**|
Source: Environment Canada. GLAP IV Total Work Plan, August 21, 2008.xls. Internal document.
* The priority groupings are based on those identified by the internal Review Committee recommendations and not those identified in the GLAP IV program documentation. The only difference is the Review Committee group 1 AOCs includes the St. Clair River and excludes the Spanish River.
** Spanish Harbour (in recovery) and Port Hope were not identified as priority (group 1 or 2) AOCs, and so are not represented in Table 1.1. None of the GLAP IV projects led by other government departments (OGDs) and Environment Canada branches between 2005 and 2009 targeted these two AOCs. Environment Canada's Science and Technology Branch studied Spanish Harbour in 2005 and 2009, and the Department's Water Quality Monitoring and Surveillance group conducted water quality monitoring there in 2008.
*** Projects can address multiple AOCs falling into both of the priority groupings.
|Environment Canada – ECB||11||11||22|
|Environment Canada – NWRI||6||5||11|
Great Lakes Sustainability Fund (GLSF)
A component of GLAP IV, the GLSF is a contribution funding mechanism to foster partnerships with other agencies and local community stakeholders to advance the goals of GLAP IV. Projects carried out by DFO and PWGSC were also funded through the GLSF.
The GLSF provides technical and financial support (up to one third of the total cost) to projects that implement remedial actions to support cleanup and restoration in four key priority areas: fish and wildlife habitat rehabilitation and stewardship, contaminated sediment assessment and remediation, innovative approaches to improve municipal wastewater effluent quality, and the elimination of non-point source pollution from agricultural sources.
GLSFfunding priority is given to those submissions undertaking remedial actions identified in the COA and in RAPs as federal government responsibilities, as well as science and monitoring activities essential to supporting the design and evaluation of these actions. First priority for funding is given to those AOCs with the greatest potential for delisting in the short to medium term (group 1 AOCs as defined in the COA), and second priority is assigned to AOCs with less potential for delisting (group 2 AOCs).
The GLSF uses a directed approach in soliciting project proposals: all proposals must link closely with intended goals and results listed in the COA, which in turn identify work required to delist AOCs. The proposal assessment criteria further include:
- the need for federal participation through legislated mandate, existing federal policy or declared federal interest (the GLSFdoes not support funding for capital or operating costs of municipal infrastructure);
- advances the completion of federal actions toward the recovery and delisting of AOCs;
- the benefit that a project has towards specific measurable achievements;
- environmental impact of a project (e.g., results of environmental assessments);
- project's actual or potential technical merits;
- application of the "polluter pays" principle, and two-thirds funding provided by other financial supporters;
- input and endorsement by review agencies and peers, including RAP Implementation Teams where projects support remedial measures;
- opportunities for technology transfer to other AOCs and the Great Lakes ecosystem;
- project's ability to promote innovation and support new technologies that feed economic growth, create new opportunities and provide long-term improvements; and
- achievement of environmental benefits at the lowest possible cost.
GLSFproposals are submitted to a multi-disciplinary review, which can include subject matter experts and AOC committee or RAP management team members, to provide external opinion on the proposed project in the form of a technical review of the proposed project and its alignment with regional priorities. A total of 267 GLSF contribution agreements worth nearly $16 million have been supported through GLAP IV since 2005, with the majority (n=158) targeting group 2 AOCs (those with less potential for delisting) – an activity focus that appears to be at odds with the stated intention of the program to focus on group 1 AOCs.
1.1.3 Stakeholders and Recipients
The achievement of GLAP IV program objectives depends on the engagement of a variety of stakeholders. As indicated above, Environment Canada is the lead department for GLAP IV, and the program is the responsibility of the Strategic Integration and Partnerships Division in the Ontario Region. Scientific support for GLAP IV is provided by internal Environment Canada partners, particularly the NWRI, the Canadian Wildlife Service and Environmental Protection Operations Directorate within the Environmental Stewardship Branch, and the Water Science and Technology Directorate within the Science and Technology Branch. Although Environment Canada has primary responsibility for the delivery of GLAP IV, the program nonetheless relies on the technical and scientific expertise of partner OGDs (other government departments) to undertake key activities for the remediation of AOCs and to provide guidance (through their participation on various oversight committees, such as GLEC, etc.) on priorities for the remediation of these areas. Federal partners include DFO, Health Canada, Agriculture and Agri-Food Canada, Transport Canada, Parks Canada Agency, Natural Resources Canada and PWGSC.
Other government jurisdictions are engaged through formal agreements: Canada and the United States are signatories to the GLWQA, and six federal departments and three provincial ministries are signatories to the COA.
Partnerships are a requirement of GLSF projects. GLSFpartners include provincial and municipal governments, Conservation Authorities, NGOs, First Nations, educational institutions, local community volunteers, agriculture, industrial and other business sectors, and academia. These organizations may be proponents of GLSF-funded projects or act as partners in GLSF projects (such as in a funding, advisory or implementation role).
Finally, the development and implementation of RAPs involves various partners, similar in type to those involved in GLSF projects. Local implementation teams or councils typically include federal representatives (e.g., DFO, Environment Canada), other government representatives (Ontario Ministry of the Environment [MOE] or Ministry of Natural Resources [MNR], municipal/regional, First Nations), Conservation Authorities, community groups and NGOs.
The Great Lakes Areas of Concern Section, Strategic Integration and Partnerships Division within Environment Canada, is responsible for the management and coordination of GLAP IV. While the program is managed in the Ontario Region under the Regional Director General, it is functionally accountable to the Department's Assistant Deputy Minister, Environmental Stewardship Branch. Further management oversight of GLAP IV is provided by the Great Lakes Environment Office, which is also within the Strategic Integration and Partnerships Division.
Several horizontal, federal/provincial and binational committees oversee and manage GLAP IV (described in more detail in Annex A under separate cover):
- GLEC: ensures Canada's commitments under the GLWQA are met through the effective and efficient delivery of GLAP IV.
- GLAPWorkplan Review Team: annual review of five year work plans submitted by federal departments, in consultation with the GLEC.
- COA Management Committee: responsible for the oversight and overall administration of the COA, it helps to coordinate federal/provincial work carried out in AOCs (as per Annex 1 of the COA) and to provide direction and decision making for AOC work.
- COA Annex Implementation Committee: coordinates development and implementation of the work planning process across multiple agencies.
- Great Lakes Binational Executive Committee: coordinates binational programs and activities (including in five shared AOCs targeted by GLAP IV).
- RAP committees: maintain working-level links to community and provincial stakeholders and ensure that environmental needs at the AOC level are addressed. Each RAPcommittee has a federal project lead who reports RAP activities directly to the COA Annex Implementation Committee.
Budget 2005 provided federal funding of $40 million ($8 million per year over five years) for GLAP IV. Table 1.4 describes the original allocation of these funds across fiscal years.
|2005 06||2006 07||2007 08||2008 09||2009 10||Total|
|1 Most full-time equivalent staff are situated in a Burlington, Ontario facility owned by Environment Canada.|
|Employee Benefit Plan||132||132||132||132||132||660|
Table 1.5 summarizes the allocation by organization. Funds were transferred from Environment Canada to relevant departments through interdepartmental settlements on the basis of approved work plans, while the work plans prepared for Environment Canada projects specified that GLAP IV resources would be allocated directly to these organizations. Information on actual GLAP IV expenditures for the remaining years of the program is not available due to gaps in financial reporting of expenditures at Environment Canada.
|Organization||Annual Recommended Funding||Total Recommended Funding 2005 2010|
|Environment Canada – ECB||1,193||9,568|
|Environment Canada – GLSF||3,784||18,918|
|Environment Canada – NWRI||1,730||8,648|
1.1.6 Program Logic Model
A logic model is a visual representation of a program/initiative that identifies the linkages between an initiative's activities and the achievement of its outcomes. Figure 1 shows the program logic model, which presents a graphical depiction of how the activities and outputs of GLAP IV relate to immediate, intermediate, long-term and ultimate outcomes.
Figure 1: Logic Model – GLAP IV
1.1.7 Performance Reporting and Evaluations
There are a number of reporting exercises related to federal Great Lakes ecosystems programs:
- GLWQA: The parties to the agreement (Canada and the United States) report biennially to the IJC on progress in developing and implementing RAPs. The Commission reports to federal, state and provincial governments every two years on progress in relation to achievement of GLWQA goals and objectives, and provides ongoing assessment and advice to governments through semi-annual meetings and special reports. RAPs for each AOC are also submitted to the Commission for review and comment at three stages:
- Stage 1 RAPs – upon completion of definition of the problem and definition of the causes of the use impairment (including a description of all involved known sources of pollutants and an evaluation of other possible sources);
- Stage 2 RAPs – submitted when remedial and regulatory measures are selected, and are to include a schedule for their implementation, as well as identification of the persons or agencies responsible for remedial measures implementation; and
- Stage 3 RAPs – submitted when monitoring indicates that identified beneficial uses have been restored.
- COA: A multi-year work plan is developed and updated annually for the COA, based on input from all the participating federal departments and provincial ministries that are parties to the COA. Reporting on activities and achievements in the work plans is analyzed and evaluated in order to ensure that key contributions are being made by participating agencies to enable achievement of COA commitments.
While the GLAP programs themselves have not been the subject of a program evaluation, other performance reporting initiatives have included:
- a mid-term review of GLAP IV;
- a review of the GLBEI by the Commissioner of the Environment and Sustainable Development (CESD) completed in 2008 (a follow-up to the 2001 CESD audit); and
- participation of the GLBEI in a Treasury Board Management Accountability Framework (TB MAF) exercise.
 The IJC was established in 1909 by the Boundary Waters Treaty between the United States and Canada. The role of the IJC is described in detail in Article VII of the GLWQA.
 The 1987 amendments also mandated the development and implementation of lakewide management plans, and committed both countries to control pollution from non-point sources, identify the nature and extent of sediment pollution, and develop methods to evaluate the impact of contaminated sediments and the technological capabilities of programs to clean them up.
 Two of these AOCs have since been delisted: Collingwood Harbour in 1994 and Severn Sound in 2003. Although officially announced on April 16, 2010, the delisting of Wheatley Harbour was not factored into the analysis of outcomes achievement, as it occurred outside the time frame for GLAP IV, which ended on March 31 2010, and subsequent to data collection and analysis for the present evaluation.
 Development and implementation of binational lake-wide management plans for the restoration, conservation and protection of the ecosystems of each of the five Great Lakes and Lake St. Clair, and implementation of the Great Lakes Binational Toxics Strategy (which deals with the development and implementation of challenge goals for the reduction of priority persistent toxic substances within the Great Lakes Basin), are the other two priorities of the GLBEI (Treasury Board Secretariat 2008 Strategic Review of GLBEI).
 The federal government has provided GLAP funding since 1989 ($125 million over 5 years in 1989; $150 million over 6 years in 1994; $40 million over 5 years in 2000; and $40 million over 5 years in 2005). Identification, evaluation and remediation of AOCs have been a major element of GLAP funding since 1989 and the single focus of GLAP funding since 2000.
 Federal actions reference the Government of Canada's actions in response to its commitments under the COA (as opposed to Ontario's). These vary by AOC.
 When it existed, the ECB included a number of divisions, four of which received GLAP IV funding: EHD (Ecosystem Health Division), RPD (Restoration Programs Division), GLSFD (Great Lakes Sustainability Fund Division), and CSD (Conservation Strategies Division). During departmental transformation, the divisions in the former ECB were restructured as follows: EHD went to Science and Technology Branch; part of CSD went to the Wildlife and Landscape Science Division within the Science and Technology Branch, while the other part of CSD became the Canadian Wildlife Service – Ontario Region (CWS‑ON) within the Environmental Stewardship Branch; and both RPD and GLSFD went to the Regional Director General's Office, and subsequently were restructured into the Great Lakes Management and Reporting Section and the Great Lakes Areas of Concern Section.
 The COA assigns federal or provincial leadership for each AOC. According to the COA Annex: a) Canada and Ontario will co-lead the RAP process in the Toronto and Region, St. Mary's River, St. Clair River, and Detroit River AOCs; b) Canada will lead the RAP process in the Thunder Bay, Hamilton Harbour, Port Hope and St. Lawrence River AOCs; and c) Ontario will lead the RAP process in the Nipigon Bay, Jackfish Bay, Peninsula Harbour, Spanish Harbour, Wheatley Harbour, Niagara River and Bay of Quinte AOCs.
 In the 2007 2010 Canada Ontario Agreement, Group 1 AOCs are Nipigon Bay, Jackfish Bay, Wheatley Harbour and St. Lawrence River (Cornwall).
 Some of these departments were invited to submit proposals for GLAP IV but declined, and were therefore not represented on the internal Review Committee. However, all OGDs listed here are members of the GLEC.
 Federal partners include Environment Canada, Parks Canada Agency (not a separate signatory, as the EC Minister is also responsible for Parks Canada), Agriculture and Agri-Food Canada, Fisheries and Oceans Canada, Health Canada, Natural Resources Canada, and Transport Canada. Provincial signatories to the agreement include the of the Environment, the Ministry of Natural Resources and the Ministry of Agriculture, Food and Rural Affairs.
 The Areas of Concern Section, Great Lakes Environment Office, and Great Lakes Management and Reporting Section now report through the Great Lakes Division.
- Date modified: