Evaluation of the Great Lakes Action Plan IV

Final Report: June 10, 2010

Previous page | Table of Contents | Next page

1.0 Introduction

In the 2009‑2010 fiscal year, Environment Canada's Audit and Evaluation Branch, Evaluation Division, commissioned an evaluation of the Great Lakes Action Plan IV (GLAP IV). This program was selected for evaluation prior to the end of the program's terms and conditions at the close of the 2009‑2010 fiscal year.

This Final Report presents the findings of the evaluation. Chapter One provides background information on GLAP IV. Chapter Two discusses the objectives of the evaluation and the methodological approach. Chapter Three presents the evaluation's findings related to relevance, design and delivery, program outcomes, cost-efficiency and cost-effectiveness. Conclusions are included in Chapter Four and recommendations in Chapter Five.

1.1 Program Profile

The five Great Lakes – Superior, Michigan, Huron, Erie and Ontario – hold one fifth of the freshwater on the Earth's surface and 80 percent of the lake and river water in North America. Approximately 30 per cent of Canadians live within the Great Lakes Basin and 45 per cent of Canada's industry is located in this area. Ensuring environmental quality in this important region has implications for the natural environment, the health and well-being of Canadians, and Canada's economic competitiveness.

The overarching mechanism for protecting and restoring the Great Lakes is the Great Lakes Water Quality Agreement (GLWQA) (1972, 1978, 1987), which is a binational agreement between Canada and the United States. The Agreement sets out a series of commitments for both countries with respect to the Great Lakes. The International Joint Commission[4] (IJC) assists the parties (Canada and the United States) in the implementation of the GLWQA. The most recent amendment to the Agreement was in 1987 and negotiations are under way for a renewed GLWQA. A key feature of the 1987 Agreement was the requirement that Canada and the United States take remedial action in heavily degraded locations or "areas of concern" (AOCs) around the lakes.[5] AOCs were identified in 1985 by the IJC and relevant federal, provincial and state governments. In total, 17 AOCs[6] were identified in Canada – twelve in Ontario and another five (along connecting channels) shared by Canada and the United States.

Canada's responsibility for managing and protecting the Great Lakes involves both federal and provincial (Ontario) jurisdictions. Both levels of government share authority to protect the environment, and are involved in aspects such as water, agriculture, species and spaces, and fisheries. Coordination of federal and provincial efforts with respect to the Great Lakes is accomplished through the Canada‑Ontario Agreement Respecting the Great Lakes Basin Ecosystem (COA). The goals and objectives of the COA parallel those of the GLWQA, and thus the Agreement includes an Annex on AOCs.

At the federal level, the Great Lakes Basin Ecosystem Initiative (GLBEI) is Environment Canada's mechanism for coordinating and delivering on federal commitments stemming from the GLWQA and the COA with respect to the Great Lakes Basin ecosystem. Remediation of domestic and binational Great Lakes AOCs is one of three priorities of the GLBEI.[7]

The Great Lakes Action Plan for Areas of Concern is the primary vehicle under the GLBEI for the federal government, with Environment Canada as the lead department, to act to restore AOCs around the Great Lakes area and fulfill Canadian commitments under the GLWQA. The current fourth phase[8] of the Action Plan, GLAP IV, was approved in 2005 with a budget of $40 million ($8 million per year over five years) to restore, protect and conserve AOCs around the Great Lakes.

Top of Page

1.1.1 Program Goals

The GLWQA states that "...the Parties, in cooperation with the State and Provincial Governments and the Commission, shall identify and work toward the elimination of ... Areas of Concern... Remedial Action Plans ... shall embody a systematic and comprehensive ecosystem approach to restoring and protecting beneficial uses in Areas of Concern... The Parties shall cooperate with State and Provincial Governments to ensure that Remedial Action Plans are developed and implemented for Areas of Concern."

The goals of GLAP IV are:

To achieve these goals, GLAP IV is based on an ecosystem approach: remedial efforts targeted to AOCs under GLAP IV deal with interrelated environmental and sustainable development issues. As laid out in the GLWQA, remediation in each AOC is guided by Remedial Action Plans (RAPs). The original RAP reports were developed in 1987 by the federal and Ontario governments, with extensive public consultation. RAP teams with scientific/technical expertise for each AOC identified the nature and causes of environmental problems, and recommended actions, implementation plans and monitoring strategies. RAP reports are issued and updated for three stages of implementation: Stage 1 – Problem definition; Stage 2 – Selection of remedial measures; and Stage 3 – Restoration of beneficial uses.

The definition and characterization of the environmental problems in the AOCs is founded on identification of the beneficial uses that are impaired, the degree of impairment and the geographic extent of such impairment. Beneficial use is described as "the ability of living organisms to use the ecosystem without adverse consequence." There are 14 beneficial use impairments (BUIs) that are assessed:

  1. Restrictions on fish consumption
  2. Tainting of fish and wildlife flavour
  3. Degradation of fish and wildlife populations
  4. Fish tumours or other deformities
  5. Bird (or other animal) deformities or reproduction problems
  6. Degradation of benthos (organisms living in or near marine sediment environments)
  7. Restrictions on dredging activities
  8. Eutrophication or undesirable algae
  9. Restrictions on drinking water consumption, or taste and odour problems
  10. Beach closings / water contact‑sports restrictions
  11. Degradation of aesthetics
  12. Added costs to agriculture or industry
  13. Degradation of phytoplankton and zooplankton populations
  14. Loss of fish and wildlife habitat

Prior to the renewal of the GLAP in 2005, a total of 98 BUIs were identified across the 15 Canadian AOCs, and 14 additional beneficial uses required further evaluation. Using this information, RAP reports were updated and the priority work required in each AOC for delisting was identified. Removal of the designation as an AOC and monitoring of recovery indicates that the identified beneficial uses are no longer impaired and the area is restored. An Area in Recovery is a designation used to indicate that all remedial actions have been taken and the area must enter a period of natural recovery.

Top of Page

1.1.2 Program Activities

Upon renewal of the GLAP in 2005, Environment Canada, with federal partners, undertook a work planning and priority-setting effort. The federal partners that participated were the National Water Research Institute (NWRI) of Environment Canada, the former Environmental Conservation Branch (ECB),[10] the Department of Fisheries and Oceans (DFO), and Public Works and Government Services Canada (PWGSC).

The recommendations of the GLAP Work Plan Review Team were approved by the federal Great Lakes Executive Committee (GLEC) and signed off by the Environment Canada GLAP program director. GLAP IV funding was distributed to Environment Canada, DFO and PWGSC in April 2006.

Priority Investments

Three broad categories of priority investments under GLAP IV within AOCs are:

Excluding GLSF projects, approximately 40 federal science and monitoring projects were recommended for implementation over the life of the program. Table 1.1 displays these projects by AOC priority grouping and Table 1.2 by investment priority. This information is drawn from the recommended work plans for federal partners; information on actual project expenditures is not known, due to gaps in financial tracking of this information within Environment Canada (discussed in section 3.2.4).

Table 1.1: Federal Partner Projects Recommended for Funding: Priority AOC Grouping*
Group 1 Priority AOCs Group 2 Priority AOCs Total**
Source: Environment Canada. GLAP IV Total Work Plan, August 21, 2008.xls. Internal document.

* The priority groupings are based on those identified by the internal Review Committee recommendations and not those identified in the GLAP IV program documentation. The only difference is the Review Committee group 1 AOCs includes the St. Clair River and excludes the Spanish River.

** Spanish Harbour (in recovery) and Port Hope were not identified as priority (group 1 or 2) AOCs, and so are not represented in Table 1.1. None of the GLAP IV projects led by other government departments (OGDs) and Environment Canada branches between 2005 and 2009 targeted these two AOCs. Environment Canada's Science and Technology Branch studied Spanish Harbour in 2005 and 2009, and the Department's Water Quality Monitoring and Surveillance group conducted water quality monitoring there in 2008.

*** Projects can address multiple AOCs falling into both of the priority groupings.
Environment Canada – ECB 11 11 22
Environment Canada – NWRI 6 5 11
PWGSC 3 3 6
DFO 8 14 22
Total 28 33 61***

Table 1.2: Number and Value of Federal GLAP IV Projects Recommended for Funding by Priority Area and Partner

Great Lakes Sustainability Fund (GLSF)

A component of GLAP IV, the GLSF is a contribution funding mechanism to foster partnerships with other agencies and local community stakeholders to advance the goals of GLAP IV. Projects carried out by DFO and PWGSC were also funded through the GLSF.

The GLSF provides technical and financial support (up to one third of the total cost) to projects that implement remedial actions to support cleanup and restoration in four key priority areas: fish and wildlife habitat rehabilitation and stewardship, contaminated sediment assessment and remediation, innovative approaches to improve municipal wastewater effluent quality, and the elimination of non-point source pollution from agricultural sources.

GLSF funding priority is given to those submissions undertaking remedial actions identified in the COA and in RAPs as federal government responsibilities, as well as science and monitoring activities essential to supporting the design and evaluation of these actions.[11] First priority for funding is given to those AOCs with the greatest potential for delisting in the short to medium term (group 1 AOCs as defined in the COA), and second priority is assigned to AOCs with less potential for delisting (group 2 AOCs).[12]

The GLSF uses a directed approach in soliciting project proposals: all proposals must link closely with intended goals and results listed in the COA, which in turn identify work required to delist AOCs. The proposal assessment criteria further include:

GLSF proposals are submitted to a multi-disciplinary review, which can include subject‑matter experts and AOC committee or RAP management team members, to provide external opinion on the proposed project in the form of a technical review of the proposed project and its alignment with regional priorities. A total of 267 GLSF contribution agreements worth nearly $16 million have been supported through GLAP IV since 2005, with the majority (n=158) targeting group 2 AOCs (those with less potential for delisting) – an activity focus that appears to be at odds with the stated intention of the program to focus on group 1 AOCs.

Table 1.3: Value and Number of GLSF Contribution Agreements, 2005‑2010

Top of Page

1.1.3 Stakeholders and Recipients

The achievement of GLAP IV program objectives depends on the engagement of a variety of stakeholders. As indicated above, Environment Canada is the lead department for GLAP IV, and the program is the responsibility of the Strategic Integration and Partnerships Division in the Ontario Region. Scientific support for GLAP IV is provided by internal Environment Canada partners, particularly the NWRI, the Canadian Wildlife Service and Environmental Protection Operations Directorate within the Environmental Stewardship Branch, and the Water Science and Technology Directorate within the Science and Technology Branch. Although Environment Canada has primary responsibility for the delivery of GLAP IV, the program nonetheless relies on the technical and scientific expertise of partner OGDs (other government departments) to undertake key activities for the remediation of AOCs and to provide guidance (through their participation on various oversight committees, such as GLEC, etc.) on priorities for the remediation of these areas. Federal partners include DFO, Health Canada, Agriculture and Agri-Food Canada, Transport Canada, Parks Canada Agency, Natural Resources Canada and PWGSC.[13]

Other government jurisdictions are engaged through formal agreements: Canada and the United States are signatories to the GLWQA, and six federal departments and three provincial ministries are signatories to the COA.[14]

Partnerships are a requirement of GLSF projects. GLSF partners include provincial and municipal governments, Conservation Authorities, NGOs, First Nations, educational institutions, local community volunteers, agriculture, industrial and other business sectors, and academia. These organizations may be proponents of GLSF-funded projects or act as partners in GLSF projects (such as in a funding, advisory or implementation role).

Finally, the development and implementation of RAPs involves various partners, similar in type to those involved in GLSF projects. Local implementation teams or councils typically include federal representatives (e.g., DFO, Environment Canada), other government representatives (Ontario Ministry of the Environment [MOE] or Ministry of Natural Resources [MNR], municipal/regional, First Nations), Conservation Authorities, community groups and NGOs.

Top of Page

1.1.4 Governance

The Great Lakes Areas of Concern Section, Strategic Integration and Partnerships Division within Environment Canada, is responsible for the management and coordination of GLAP IV.[15] While the program is managed in the Ontario Region under the Regional Director General, it is functionally accountable to the Department's Assistant Deputy Minister, Environmental Stewardship Branch. Further management oversight of GLAP IV is provided by the Great Lakes Environment Office, which is also within the Strategic Integration and Partnerships Division.

Several horizontal, federal/provincial and binational committees oversee and manage GLAP IV (described in more detail in Annex A under separate cover):

Top of Page

1.1.5 Resources

Budget 2005 provided federal funding of $40 million ($8 million per year over five years) for GLAP IV. Table 1.4 describes the original allocation of these funds across fiscal years.

Table 1.4: Allocation of GLAP IV Funds ($000s)
2005‑06 2006‑07 2007‑08 2008‑09 2009‑10 Total
1 Most full-time equivalent staff are situated in a Burlington, Ontario facility owned by Environment Canada.
Salary1 659 659 659 659 659 3,295
Employee Benefit Plan 132 132 132 132 132 660
Operating 7,209 7,209 7,209 7,209 7,209 36,045
Total 8,000 8,000 8,000 8,000 8,000 40,000

Table 1.5 summarizes the allocation by organization. Funds were transferred from Environment Canada to relevant departments through interdepartmental settlements on the basis of approved work plans, while the work plans prepared for Environment Canada projects specified that GLAP IV resources would be allocated directly to these organizations. Information on actual GLAP IV expenditures for the remaining years of the program is not available due to gaps in financial reporting of expenditures at Environment Canada.

Table 1.5: Summary of Work Plans – Recommended Funding ($000s)
Organization Annual Recommended Funding Total Recommended Funding 2005‑2010
DFO 573 2,866
Environment Canada – ECB 1,193 9,568
Environment Canada – GLSF 3,784 18,918
Environment Canada – NWRI 1,730 8,648
Total 8,000 40,000

Top of Page

1.1.6 Program Logic Model

A logic model is a visual representation of a program/initiative that identifies the linkages between an initiative's activities and the achievement of its outcomes. Figure 1 shows the program logic model, which presents a graphical depiction of how the activities and outputs of GLAP IV relate to immediate, intermediate, long-term and ultimate outcomes.

Figure 1: Logic Model – GLAP IV

This figure shows the program logic model, which presents a graphical depiction of how the activities and outputs of the Great Lakes Action Plan IV relate to immediate, intermediate, long-term and ultimate outcomes.
Click to enlarge

Top of Page

1.1.7 Performance Reporting and Evaluations

There are a number of reporting exercises related to federal Great Lakes ecosystems programs:

While the GLAP programs themselves have not been the subject of a program evaluation, other performance reporting initiatives have included:

[4] The IJC was established in 1909 by the Boundary Waters Treaty between the United States and Canada. The role of the IJC is described in detail in Article VII of the GLWQA.

[5] The 1987 amendments also mandated the development and implementation of lakewide management plans, and committed both countries to control pollution from non-point sources, identify the nature and extent of sediment pollution, and develop methods to evaluate the impact of contaminated sediments and the technological capabilities of programs to clean them up.

[6] Two of these AOCs have since been delisted: Collingwood Harbour in 1994 and Severn Sound in 2003. Although officially announced on April 16, 2010, the delisting of Wheatley Harbour was not factored into the analysis of outcomes achievement, as it occurred outside the time frame for GLAP IV, which ended on March 31 2010, and subsequent to data collection and analysis for the present evaluation.

[7] Development and implementation of binational lake-wide management plans for the restoration, conservation and protection of the ecosystems of each of the five Great Lakes and Lake St. Clair, and implementation of the Great Lakes Binational Toxics Strategy (which deals with the development and implementation of challenge goals for the reduction of priority persistent toxic substances within the Great Lakes Basin), are the other two priorities of the GLBEI (Treasury Board Secretariat 2008 Strategic Review of GLBEI).

[8] The federal government has provided GLAP funding since 1989 ($125 million over 5 years in 1989; $150 million over 6 years in 1994; $40 million over 5 years in 2000; and $40 million over 5 years in 2005). Identification, evaluation and remediation of AOCs have been a major element of GLAP funding since 1989 and the single focus of GLAP funding since 2000.

[9] Federal actions reference the Government of Canada's actions in response to its commitments under the COA (as opposed to Ontario's). These vary by AOC.

[10] When it existed, the ECB included a number of divisions, four of which received GLAP IV funding: EHD (Ecosystem Health Division), RPD (Restoration Programs Division), GLSFD (Great Lakes Sustainability Fund Division), and CSD (Conservation Strategies Division). During departmental transformation, the divisions in the former ECB were restructured as follows: EHD went to Science and Technology Branch; part of CSD went to the Wildlife and Landscape Science Division within the Science and Technology Branch, while the other part of CSD became the Canadian Wildlife Service – Ontario Region (CWS‑ON) within the Environmental Stewardship Branch; and both RPD and GLSFD went to the Regional Director General's Office, and subsequently were restructured into the Great Lakes Management and Reporting Section and the Great Lakes Areas of Concern Section.

[11] The COA assigns federal or provincial leadership for each AOC. According to the COA Annex: a) Canada and Ontario will co-lead the RAP process in the Toronto and Region, St. Mary's River, St. Clair River, and Detroit River AOCs; b) Canada will lead the RAP process in the Thunder Bay, Hamilton Harbour, Port Hope and St. Lawrence River AOCs; and c) Ontario will lead the RAP process in the Nipigon Bay, Jackfish Bay, Peninsula Harbour, Spanish Harbour, Wheatley Harbour, Niagara River and Bay of Quinte AOCs.

[12] In the 2007‑2010 Canada‑Ontario Agreement, Group 1 AOCs are Nipigon Bay, Jackfish Bay, Wheatley Harbour and St. Lawrence River (Cornwall).

[13] Some of these departments were invited to submit proposals for GLAP IV but declined, and were therefore not represented on the internal Review Committee. However, all OGDs listed here are members of the GLEC.

[14] Federal partners include Environment Canada, Parks Canada Agency (not a separate signatory, as the EC Minister is also responsible for Parks Canada), Agriculture and Agri-Food Canada, Fisheries and Oceans Canada, Health Canada, Natural Resources Canada, and Transport Canada.  Provincial signatories to the agreement include the of the Environment, the Ministry of Natural Resources and the Ministry of Agriculture, Food and Rural Affairs.

[15] The Areas of Concern Section, Great Lakes Environment Office, and Great Lakes Management and Reporting Section now report through the Great Lakes Division.

Previous page | Table of Contents | Next page