This page has been archived on the Web

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Skip booklet index and go to page content

Evaluation of the Great Lakes Action Plan IV

Report Clearance Steps

Planning phase completed

December 2009

Report completed

March 31, 2010

Report sent for management response

April 29, 2010

Management response received

May 25, 2010

Report approved by Departmental Evaluation Committee (DEC)

June 10, 2010

Top of Page

abbrs used in the report

AOC
Areas of Concern
BUI
Beneficial use impairments
CESD
Commissioner of the Environment and Sustainable Development
COA
Canada-Ontario Agreement Respecting the Great Lakes Basin Ecosystem
DFO
Fisheries and Oceans Canada
ECB
Environmental Conservation Branch
GLAP
Great Lakes Action Plan
GLAP IV
Great Lakes Action Plan for Areas of Concern (Phase IV)
GLBEI
Great Lakes Basin Ecosystem Initiative
GLEC
Great Lakes Executive Committee
GLSF
Great Lakes Sustainability Fund
GLWQA
Great Lakes Water Quality Agreement
G&C
Grants and Contributions
IJC
International Joint Commission
MC
Management Committee
MNR
Ministry of Natural Resources (Ontario)
MOE
Ministry of the Environment (Ontario)
MOU
Memorandum of understanding
NGOs
Non-governmental organizations
NWRI
National Water Research Institute
OGDs
Other government departments
O&M
Operations and maintenance
PWGSC
Public Works and Government Services Canada
RAP
Remedial Action Plan
RDG
Regional Director General
TB MAF
Treasury Board Management Accountability Framework

Acknowledgments

The Audit and Evaluation Branch, Evaluation Division Project Team led by William Blois, under the direction of the Director, Shelley Borys, would like to thank those individuals who contributed to this project, particularly members of the Evaluation Committee as well as all interviewees who provided insights and comments crucial to this evaluation. This evaluation was conducted and reported by EKOS Research Associates Limited.

Return to Table of Contents

Executive Summary

Executive Summary

Background

In the 2009-2010 fiscal year, Environment Canada's Audit and Evaluation Branch, Evaluation Division, commissioned an evaluation of the Great Lakes Action Plan IV (GLAP IV). This program was selected for evaluation prior to the end of the program's terms and conditions at the close of the 2009-2010 fiscal year.

The GLAP for Areas of Concern (AOCs) is the primary vehicle under the Great Lakes Basin Ecosystem Initiative (GLBEI) for the federal government, with Environment Canada as the lead department, to act to restore AOCs around the Great Lakes area and fulfill Canadian commitments under the Great Lakes Water Quality Agreement (GLWQA) with the United States. The current fourth phase[1] of the Action Plan, GLAP IV, was approved in 2005 with a budget of $40 million ($8 million per year over five years) to restore, protect and conserve AOCs around the Great Lakes.

The goals of GLAP IV are:

  • to make progress on federal[2] actions in eight AOCs: St. Mary's River, St. Clair River, Toronto and Region, Hamilton Harbour, Jackfish Bay, Detroit River, Niagara River and Port Hope; and
  • to complete federal actions in seven AOCs: Thunder Bay, Nipigon Bay, Peninsula Harbour, Bay of Quinte, Wheatley Harbour, St. Lawrence River and Spanish River.

To achieve these goals, GLAP IV is based on an ecosystem approach: remedial efforts targeted to AOCs under GLAP IV deal with interrelated environmental and sustainable development issues based on beneficial use impairments (BUIs) identified in these areas. As laid out in the GLWQA, remediation in each AOC is guided by Remedial Action Plans (RAPs).

Program funding is allocated to remedial actions (e.g., sediment remediation, habitat restoration), science (e.g., assessment and monitoring of ecosystem health and status of BUIs), and engagement/governance (e.g., engagement of community members, support for RAP committees). Over the life of the program, approximately 40 science and monitoring projects led by federal partners and over 200 grants and contributions (G&C) projects under the Great Lakes Sustainability Fund (GLSF) were approved for funding.

Evaluation Issues

The evaluation covers GLAP IV program activities from the 2005 2006 to 2009 2010 fiscal years. The evaluation issues include the following:

  • Relevance: Is there a continued need for a GLAP? Is GLAP IV aligned to federal government priorities? Is GLAP IV consistent with federal roles and responsibilities?
  • Design and Delivery: Is the GLAP IV design appropriate for achieving expected program results?
  • Achievement of program outcomes: To what extent have intended outcomes been achieved as a result of GLAP IV? Have there been any unintended (positive or negative) outcomes?
  • Efficiency: Is GLAP IV undertaking activities and delivering products in the most efficient manner?
  • Economy: Is GLAP IV achieving its intended outcomes in the most economical manner?

Methodology

Data were collected for the evaluation using multiple lines of evidence. These included a document review, a review of 39 GLSF G&C agreement files, a qualitative analysis of 21 federal partner project files, and a total of 46 key informant interviews with departmental program managers and federal partners (n=11); representatives of various committees (n=10); federal project proponents (n=4); GLSF project proponents (n=10); unsuccessful applicants (n=4); RAP committee members (n=4); and international or academic experts on aquatic ecosystem remediation (n=3). Key challenges for the study included a reliance on internal sources of evidence and limited availability of program activity information and financial data with respect to GLAP IV.

Top of Page

Evaluation Findings

Relevance

GLAP IV remains a relevant program that enables the federal government to address the continued need for restoration and maintenance of the Great Lakes AOCs. While recent trends in Great Lakes' ecosystem conditions are variable, historical sources of stress, combined with new challenges, are leading to negative impacts in many areas of the lakes. Environmental need, together with the social and economic benefits of the Great Lakes, supports the continued relevance of the program. Public opinion is consistent with scientific information and key informant views on the environmental and societal importance of the Great Lakes and the continued need for the GLAP.

There is little redundancy risk associated with GLAP IV. The program uniquely addresses federal commitments in the GLWQA and the Canada Ontario Agreement Respecting the Great Lakes Basin Ecosystem (COA) by providing a framework to address BUIs linked to AOC RAP priority areas. Unlike other public funding (federal, provincial and municipal) and non-public funding (industry, foundations, non governmental organizations [NGOs], etc.), GLAP IV is targeted exclusively to AOCs. Key informants express few concerns about the possibility of duplication or overlap between GLAP IV and other programming. Rather, GLAP IV is seen as being highly complementary and as a catalyst in orienting other funding sources to provide support in the AOCs.

GLAP IV is consistent with federal and departmental priorities. The program is part of the GLBEI and is one of six ecosystem initiatives at Environment Canada under the Priority Ecosystem Initiatives Sub-Activity. The GLBEI supports the Department's Strategic Outcome that "Canada's natural capital is restored, conserved and enhanced." GLAP IV outcomes and priority investments also support a number of other sub-activities within Environment Canada's program activity architecture, including Aquatic Ecosystems, Wildlife, Priority Ecosystems, Ecosystems Sustainability, and Assessment and Ecological Monitoring.

Environment Canada's GLAP IV is aligned to support federal government commitments and obligations under the GLWQA and COA, as well as federal government priorities outlined in the 2007 Speech from the Throne and two recent federal budgets. Legislative authorities for the GLAP IV include the Canada Water Act, the International Boundary Waters Treaty Act, and the Canadian Environmental Protection Act.

Design and Delivery

Overall, GLAP IV was viewed as a sensible model to achieve intended outcomes. Engagement of partners represents a critical and beneficial aspect of the GLAP IV program delivery, with partnerships occuring at many levels and in many ways. Formal agreements between Canada and the United States (GLWQA) and Canada and Ontario (COA) represent the foundation for commitments of the jurisdictional parties with respect to the AOCs. Implementation involves horizontal partnerships within Environment Canada and across the federal government. RAP committees provide a forum for jurisdictions (federal, provincial, municipal and First Nations) and key stakeholder groups (conservation authorities, community groups and NGOs) to coordinate their efforts through to the delisting process. The funding structure of GLSF projects promotes partner engagement at the local and regional level with partner contributions including cash and in-kind funding, assistance with delivery, and participation in advisory or research capacity.

The implementation of GLAP IV diverged from the original design of the program in several areas. Although the five year work plans required by federal partner departments to access GLAP IV funds were considered to be a positive and worthwhile exercise, the work planning, annual reporting, review and updating processes did not occur as intended, which resulted in a loss of flexibility, responsiveness and accountability.

Due to the horizontal nature of the initiative and broad changes at Environment Canada, GLAP IV governance and management activities do not reflect those outlined in the management framework developed for the program. Areas identified as a potential source of confusion or concern by federal key informants include an inadequate instrument to ensure that partner contributions to program goals are fulfilled, and clarity of roles and responsibilities with respect to delisting AOCs.

For federal partners who received their recommended funding, the amount of GLAP IV funding was seen to be adequate, with the caveat that they would like future funding to be indexed for inflation. However, some federal partners report receiving significantly fewer resources than their original approved funding levels, which they feel impacted negatively on their ability to implement planned activities. For federal partners that received their GLAP IV funding allocation in full, a more pressing concern was the high ratio of operations and maintenance (O&M) to salary dollars and inability to convert these dollars (i.e., from O&M to salary), which left many groups with insufficient salary dollars to hire scientific and technical staff. GLSF project proponent and committee key informants tended to suggest that the GLAP IV focus and resources be expanded to include priority areas and activities that extend beyond the designated AOCs.

GLAP IV monitoring and reporting activities are undertaken to meet annual reporting obligations associated with both the GLWQA and the COA. At the level of funded projects, GLSF project reporting is quite complete and the majority of reviewed GLSF files provided clear evidence of intended output achievements, however, limited evidence was provided on the outcomes of remedial actions taken in the AOCs. While federal proponents were supportive of increasing reporting requirements for GLAP IV projects to enhance performance measurement and accountability, regular reporting of federal science and monitoring projects is not a formal requirement and reporting to the GLAP IV program is ad hoc.

Project proponent key informants indicate a high level of satisfaction with the GLSF program. Program priorities are considered to be clear and appropriate. The application process is regarded as clear and transparent by most funding recipients and selection criteria are viewed to be explicit and well communicated. Overall, the GLSF proposal review process is perceived to be logical and GLSF funding decisions are viewed as strategic and fair. Program staff are seen by project proponents to be accessible and supportive. Modifications to the GLSF program (including a shift from the use of memoranda of understanding to contribution agreements), implemented by Environment Canada in 2007, have challenged the timely delivery of the program, reduced flexibility and increased uncertainty among partners. Project proponents indicate a preference for streamlining the application and approval process, for multi-year agreements and for more straightforward funding agreements.

Achievement of Program Outcomes

In general, key informants had favourable impressions on the performance of the program with respect to the achievement of immediate and intermediate outcomes, a finding that is supported by the program documentation and review of GLSF and federal project files. As indicated, engagement of partners at the local level and across jurisdictions and scientific communities is perceived to be a strength of the program. This is supported by GLAP IV funds allocated to federal partners to facilitate coordination and management of GLAP IV (e.g., with the provinces and First Nations). Engagement of partners and participation at the local level is facilitated through committees, informal networks and the RAP structure (which also receive GLSF funding support). With respect to remedial actions, addressing pollution (through sediment remediation, support to municipal infrastructure improvements and reducing non-point sources of pollution) is a significant priority for the program and is allocated almost one half of the GLAP IV dollars. While the success of federal partners' efforts is difficult to discern due to gaps in the federal project files, GLSF projects that supported this objective were found to be well documented in terms of outputs and experienced few challenges in implementation. BUI assessment and monitoring work is occurring on many fronts to assess the status of BUIs, effectiveness of restoration activities and defining BUI goals/targets. Fewer federal projects were funded for habitat restoration (though this is a particular focus of GLSF). Habitat projects are more easily documented in terms of outputs (e.g., plantings, wetland acreage), but these projects are also more subject to implementation challenges. Overall, program investments advance activity in each AOC to some degree (though some coordination projects target all AOCs generically). Note that while group 1 AOCs (those closest to delisting) were intended to be assigned a higher priority for GLAP IV investments, the number of projects funded in these group 1 AOCs is lower than in the group 2 AOCs.

Achievement of the program's longer-term outcomes is much less evident, and the majority of BUIs that were originally identified in AOCs still exist. Of the over 100 BUIs identified, only 20 per cent have been restored. In consideration of the status of the BUIs, the program has identified outstanding priority actions for each AOC and projected timelines for delisting. Only two AOCs have been delisted and one AOC designated an area in recovery and these occurred prior to GLAP IV. The original program goal of completing federal actions in seven group 1 AOCs has not been achieved. Three AOCs are expected to be delisted or designated as Areas in Recovery within the next one to two years, and a fourth, Wheatley Harbour, has just been delisted (announced on April 16, 2010). With respect to the remaining AOCs, the picture is much more complex and the time frames for delisting are longer-term, with most of these AOCs estimating delisting as occurring between 2015 and 2020. For committee and expert key informants, the overall importance of the contribution made by GLAP IV to the restoration and maintenance of AOCs is viewed as being very significant, if not critical.

For key informants, positive external factors that support program success include: leveraged funding, particularly infrastructure funding; United States investments; and community/political engagement (e.g., the Great Lakes and St. Lawrence Cities Initiative). External factors that interviewees identified as having the potential to negatively affect the success of GLAP IV include: new and changing ecosystem issues; and the economic downturn. Unintended outcomes of GLAP IV cited by key informants tend to be positive and focus on unexpected interest and engagement of non-targeted groups (e.g., landowners) and the general public, as well as on unanticipated opportunities for collaboration and knowledge transfer.

Efficiency

The cost-efficiency of GLAP IV is difficult to determine with existing information due to the unbundling[3] of GLBEI funds and activities and the introduction of a shared, results based accountability approach to environmental initiatives. The impact on financial data availability resulting from the changes implemented to Environment Canada's resource allocation and accountability structure is that GLAP IV funds are not specifically coded at the program level and the A-base support allocated to GLAP IV is not known.

Financial analysis of the GLSF program component indicates that for every contribution dollar, $0.24 is spent on program administration (salary and O&M). This amount compares favourably to Environment Canada's EcoAction Community Funding Program, which has a cost-efficiency ratio of $0.39, but unfavourably with the Habitat Stewardship Program ($0.08) and the Invasive Alien Species Partnership Program ($0.13). Like EcoAction, the delivery of the GLSF involves additional staff responsibilities such as participation in RAP committees.

For key informants, there is an impression of efficient program delivery both overall and at the project level, with a number of factors identified that support cost efficiency at the program and project levels. Some suggestions to improve efficiency were nonetheless received.

Economy

There is consensus among key informants that GLAP IV is a good investment of public funds and that results are achieved in an economical way. Design aspects that are seen as contributing to the program's cost-effectiveness include: a focused effort on AOCs; cost-sharing among jurisdictions and key stakeholders; local involvement and delivery; and continuity of intervention and knowledge acquisition over successive GLAP programs. A few key informants also noted the economic benefits of remediation, as well as the cost of not intervening.

No viable alternative approaches to achieve the same or better outcomes at a lower cost were identified. The prevailing sentiment among key informants is to "stay the course" in that GLAP IV provides a solid foundation for work in the AOCs. Fundamental changes to the program were discouraged by key informants as they may jeopardize the impact of work completed to date.

Top of Page

Recommendations

Recommendations for the current or future iterations of GLAP are based on the findings and conclusions of the evaluation. Overall, GLAP IV continued to be a relevant program to address ongoing needs for ecological restoration in the AOCs and is well-aligned with departmental and federal priorities. At the end of its five-year term, GLAP IV has not fully achieved its intended longer-term outcomes in the areas of addressing BUIs and delisting AOCs, due in large part to the complexity and long-term nature of the task. The following recommendations focus on improvements to the management of the program in three areas: delivery and oversight; financial management and accountability; and performance measurement and reporting.

Delivery and Oversight

Oversight of GLAP IV was challenged by several changes that occurred within Environment Canada during the program's five year term. The changes affected both the overall management and the coordination of the program, and the GLSF. One recommendation pertains to improving coordination and accountability of the program, and one pertains to ensuring the timeliness and responsiveness of the GLSF.

1) The Regional Director General (RDG) Ontario should develop a suitable instrument to enhance coordination of federal partners involved in GLAP and the accountability of their efforts toward achieving GLAP goals. GLAP IV is a complex initiative that requires the contribution of many federal partners to achieve program goals. With the demise of the work plan review teams and process, GLAP IV lacked the instruments to ensure that these efforts were coordinated and that federal partners were implementing work plans to achieve program goals. The work planning process, convened annually as it was envisioned, could be re-instituted to enhance coordination and accountability.

2) The RDG Ontario should explore means of streamlining both the GLSF funding approvals process and the GLSF application process. While mostly satisfied with the GLSF program, proponents identified two issues with respect to administration: timeliness of approvals and an overly complex application process. Consideration should be given to scrutinizing the timing of the funding cycle to ensure that funding approvals coincide with seasonal activities, and that the application process is streamlined to promote efficiency (perhaps by exploring adoption/applicability of common application forms/guidelines being developed as part of the wider federal G&C reform exercise). Although most contribution agreements are single-year only, the majority of projects are of an ongoing nature. The use of multi-year agreements may be appropriate in some of these instances.

Financial Management and Accountability

There is limited information on GLAP IV program expenditures. Financial tracking of GLAP IV funds was weak during this iteration of the program due to limited use of codes for expenditures at the program level. One recommendation pertains to financial monitoring of GLAP funds at Environment Canada.

3) The RDG Ontario should explore ways to enhance the financial accountability of the program. The direct transfer of funds to internal partners and absence of program-specific coding of expenditures within the Department has weakened financial accountability of the program. Diligent coding of expenditures at the program level is a key element to ensure transferred funds are received and utilized within Environment Canada by departmental partners to meet program goals and priorities. The merit and potential disadvantages of direct transfers of program funds within Environment Canada to departmental partners for financial accountability should be examined.

Performance Measurement and Reporting

GLAP IV monitoring and reporting was not guided by a performance measurement framework. As a result, a broad spectrum of performance measures for federal partners and the GLSF were not identified early in the program and reporting requirements, particularly for federal partners, were not clearly articulated. Five recommendations pertain to improving performance measurement and reporting.

4) The RDG Ontario should develop a more regular and robust reporting approach for GLAP projects conducted by federal partners. Performance reporting for projects conducted by federal partners was unspecified and sporadic. A more robust reporting approach should be based on clear terms and conditions for reporting on funds allocated to federal partners. Reporting should link activities/deliverables to approved work plans, recognizing that, in some cases, federal projects are undertaken as part of an integrated science program.

5) The RDG Ontario should develop a performance measurement framework for the program, and include a spectrum of more sensitive performance measures (shorter- and longer-term). The program does not have a performance measurement framework and is not currently well positioned to demonstrate progress and interim results because measures such as delisting AOCs are long-term. A more nuanced approach must balance the benefit of performance measurement with the cost of monitoring and assessing BUIs in the AOCs. Recent efforts by the program to document accomplishments and assign roles, responsibilities and timelines for outstanding actions and priorities are acknowledged as a strong foundation for this ongoing work.

6) The RDG Ontario should support the development of information management tools to enhance implementation and monitoring of GLSF and federal projects. No systematic process exists for monitoring and reporting the activities, outputs and performance for GLAP-funded projects or the program overall. A Department-wide Management Information System for Environment Canada's community funding programs, including the GLSF, is in development, although this system is not expected to be implemented until 2012. Opportunities for interim tracking of activities and project and partner contributions should be identified.

7) The RDG Ontario should support the development of information-sharing tools to facilitate broader access to the research generated by GLAP IV by the scientific community and the public. There is no ongoing mechanism available for technical transfer of research and results across the AOCs. This is an outstanding commitment in Annex 4 of the COA.

8) The RDG Ontario should continue efforts to define delisting criteria for AOCs and clarify responsibilities with respect to delisting decision making. Roles and responsibilities for aspects of delisting AOCs are a grey area and may be variable across the AOCs. As more AOCs approach restoration, clarifying the steps to delisting will increase in importance.

Top of Page

Management Response

The RDG Ontario accepts the evaluation and all of its recommendations, and the Great Lakes Division has provided a plan to implement the following management actions in response to the evaluation recommendations, within the context of the program's renewal.

1. The RDG Ontario should develop a suitable instrument to enhance coordination of federal partners involved in the GLAP and the accountability of their efforts toward achieving GLAP goals. GLAP IV is a complex initiative that requires the contribution of many federal partners to achieve program goals. With the demise of the work plan review teams and process, GLAP IV lacked the instruments to ensure that these efforts were coordinated and that federal partners were implementing work plans to achieve program goals. The work planning process, convened annually as it was envisioned, could be re-instituted to enhance coordination and accountability.

The RDG Ontario agrees with this recommendation.

A GLAP five-year work plan will be developed by Environment Canada, Great Lakes Division, with input from all federal departments and Responsibility Centres within Environment Canada contributing to the restoration of Great Lakes AOCs and accessing GLAP funds. The GLAP Work Plan Review Team will be re-established and will annually conduct a review of progress achieved, identify actions required to complete the restoration of AOCs, determine priorities for the coming year, and propose adjustment of the work plan and funding allocations as required. The GLAP Work Plan Review Team will be led by the Great Lakes Division of Environment Canada and will comprise representatives of Environment Canada, the Department of Fisheries and Oceans, Public Works and Government Services Canada and other federal government departments as necessary. Recommendations of the GLAP Work Plan Review Team will be presented to the Director, Great Lakes Division, for approval. A GLAP work planning template will be established specifying the project rationale, (i.e., how the proposed project contributes to AOC delisting), scope of work, deliverables, schedule and resource requirements.

Timeline

Deliverable(s)

Responsible Party

August 2010

GLAP Work Plan Review Team established

Director, Great Lakes Division

October 2010

Establish GLAP work planning and reporting template

Director, Great Lakes Division

November 2010

Five-year work plans for all federal departments and groups within Environment Canada accessing GLAP funds contributing to the remediation of AOCs

Director, Great Lakes Division

March-April 2011, 2012, 2013, 2015

Review and revision of GLAP Work Plan, Annual Reports on Results, funding allocation decision

Director, Great Lakes Division; and GLAP funded groups

Top of Page

2. The RDG Ontario should explore means of streamlining both the GLSF funding approvals process and the GLSF application process. While mostly satisfied with the GLSF program, proponents identified two issues with respect to administration: timeliness of approvals and an overly complex application process. Consideration should be given to scrutinizing the timing of the funding cycle to ensure that funding approvals coincide with seasonal activities, and that the application process is streamlined to promote efficiency (perhaps by exploring adoption/applicability of common application forms/guidelines being developed as part of the wider federal G&C reform exercise). Although most contribution agreements are single-year only, the majority of projects are of an ongoing nature. The use of multi-year agreements may be appropriate in some of these instances.

The RDG Ontario agrees with this recommendation.

Environment Canada's Action Plan to Reform the Administration of Grants and Contributions will improve the efficiency of the GLSF application and approval process.

In addition, improvements have been made to the GLSF application and proposal evaluation processes to simplify and improve the efficiency and consistency of the GLSF application and approval process. Multi-year agreements will be used where appropriate.

Timeline

Deliverable(s)

Responsible Party

June 2010

Adopt application and reporting processes from the Departmental Action Plan to Reform the Administration of Grants and Contributions

Director, Great Lakes Division

Top of Page

3. The RDG Ontario should explore ways to enhance the financial accountability of the program. The direct transfer of funds to internal partners and absence of program-specific coding of expenditures within the Department has weakened financial accountability of the program. Diligent coding of expenditures at the program level is a key element to ensure transferred funds are received and utilized within Environment Canada by departmental partners to meet program goals and priorities. The merit and potential disadvantages of direct transfers of program funds within Environment Canada to departmental partners for financial accountability should be examined.

The RDG Ontario agrees with this recommendation.

The Great Lakes Division will take steps to ensure that departmental resources expended in the delivery of the GLAP are adequately documented. To this end, the Great Lakes Division will take the necessary steps to adopt program‑specific authority codes for all Environment Canada internal salary, O&M and G&C expenditures of GLAP funds. Furthermore, all GLAP funds will be managed by the Great Lakes Division and distributed to other federal departments and Responsibility Centres within Environment Canada in accordance with the GLAP five‑year work plan annual updates.

Timeline

Deliverable(s)

Responsible Party

May 2010 (done)

Request for the establishment of a GLAP‑specific authority code in the Financial Information System

Director, Great Lakes Division

May 2010 (done)

Establish program‑specific authority codes for GLAP salary, O&M and G&C funds

Environment Canada Finance Directorate

Top of Page

4. The RDG Ontario should develop a more regular and robust reporting approach for GLAP projects conducted by federal partners. Performance reporting for projects conducted by federal partners was unspecified and sporadic. A more robust reporting approach should be based on clear terms and conditions for reporting on funds allocated to federal partners. Reporting should link activities/deliverables to approved work plans, recognizing that, in some cases, federal projects are undertaken as part of an integrated science program.

The RDG Ontario agrees with this recommendation.

Environment Canada, Great Lakes Division, will require annual reports on all work plan activities from Environment Canada and other federal partners participating in the restoration of Great Lakes AOCs. Reporting information will be considered in the annual review and revision of work plans. Clear guidelines will be established for monitoring and reporting of GLAP funds by Environment Canada and federal partners. This will include the establishment of a short reporting template, including financial reporting, and pertinent performance information.

Timeline

Deliverable(s)

Responsible Party

October 2010

Establish guidelines for monitoring and reporting of GLAP funds; GLAP work reporting template established

Director, Great Lakes Division

March April 2011, 2012, 2013, 2015

An annual report on the status of GLAP work plan commitments will be prepared and reviewed through the annual GLAP work planning process

Director, Great Lakes Division

Top of Page

5. The RDG Ontario should develop a performance measurement framework for the program, and include a spectrum of more sensitive performance measures (shorter- and longer-term). The program does not have a performance measurement framework and is not currently well‑positioned to demonstrate progress and interim results because measures such as delisting AOCs are long-term. A more nuanced approach must balance the benefit of performance measurement with the cost of monitoring and assessing BUIs in the AOCs. Recent efforts by the program to document accomplishments and assign roles, responsibilities and timelines for outstanding actions and priorities are acknowledged as a strong foundation for this ongoing work.

The RDG Ontario agrees with this recommendation.

A performance measurement framework will be developed that will consider feasible measures to provide a more detailed and incremental assessment of progress toward restoration of BUIs and delisting of AOCs. An Assessment of the Status of Remaining Actions to Delist AOCs has been completed and will serve as an activity based, short term performance measurement framework. A Beneficial Use Impairment Status and Progress Report has also been completed and provides a longer‑term, result‑based, performance measurement framework. The Assessment and Report will be reviewed, revised and reported on biennially in alternating years.

Timeline

Deliverable(s)

Responsible Party

March 2011

Performance measurement framework developed for the program

Director, Great Lakes Division

February 2012, February 2014

Updated Beneficial Use Impairment Status and Progress Report

Director, Great Lakes Division

February 2011, February 2013, February 2015

Updated Assessment of the Status of Remaining Actions to Delist AOCs

Director, Great Lakes Division

Top of Page

6. The RDG Ontario should support the development of information management tools to enhance implementation and monitoring of GLSF and federal projects. No systematic process exists for monitoring and reporting the activities, outputs and performance for GLAP-funded projects or the program overall. A Department-wide Management Information System for Environment Canada's community funding programs, including the GLSF, is in development, although this system is not expected to be implemented until 2012. Opportunities for interim tracking of activities and project and partner contributions should be identified.

The RDG Ontario agrees with this recommendation.

Environment Canada's Action Plan to Reform the Administration of Grants and Contributions (the Department's G&C reform initiative) is developing an online application and information management system for G&C programs. This system will improve program efficiency, enhance alignment with departmental priorities and improve the ability to report collectively on the results of departmental funding programs. The system is scheduled to be in operation by 2011‑2012. In the interim, the Great Lakes Division has established an Excel‑based system to track GLSF project proposals, proposal reviews and selection, project recipients and contribution agreements, and project financials, products and outcomes.

Timeline

Deliverable(s)

Responsible Party

2010

Adopt information management system from the Departmental Action Plan to Reform the Administration of Grants and Contributions.

Director, Great Lakes Division

Top of Page

7. The RDG Ontario should support the development of information-sharing tools to facilitate broader access to the research generated by GLAP IV by the scientific community and the public. There is no ongoing mechanism available for technical transfer of research and results across the AOCs. This is an outstanding commitment in Annex 4 of the COA.

The RDG Ontario agrees with this recommendation.

Information on AOCs is provided through the Environment Canada website and other communications products. Research findings are regularly published in scientific journals and presented at scientific forums, including the Great Lakes State of the Lakes Ecosystem Conference and the International Association for Great Lakes Research Conference.

Additionally, AOC Progress Reports have been developed jointly with the Province of Ontario for release in 2010. This tool will be built upon and modified to publicly report on progress regarding remediation of Great Lakes AOCs, and to enhance sharing of research results and other information. Reporting on AOCs will be implemented on a three year cycle, consistent with other Great Lakes reporting.

Timeline

Deliverable(s)

Responsible Party

2013

Area of Concern Progress Reports will be issued every three years and will communicate research results to the scientific community and general public

Director, Great Lakes Division

Top of Page

8. The RDG Ontario should continue efforts to define delisting criteria for AOCs and clarify responsibilities with respect to delisting decision making. Roles and responsibilities for aspects of delisting AOCs are a grey area and may be variable across the AOCs. As more AOCs approach restoration, clarifying the steps to delisting will increase in importance.

The RDG Ontario agrees with this recommendation.

The GLWQA clearly states that the parties (Canada and the United States) are responsible for leading all aspects of the AOC process "in cooperation with State and Provincial Governments."

Environment Canada's January 2010 Great Lakes RAP Workshop, which included federal and provincial representatives involved in Great Lakes AOCs as well as the local RAP coordinators, addressed the delisting process issue as an agenda item. The principle outcome was a commitment to develop a Canada Ontario Guide to the Designation of Beneficial Use Impairments, Preparing Stage 3 Reports and Delisting Areas of Concern. The guide, which is being developed collaboratively with the Ontario Ministry of the Environment, will clarify the responsibilities with respect to delisting decision making.

Timeline

Deliverable(s)

Responsible Party

December 2010

Canada–Ontario Guide to the Designation of Beneficial Use Impairments, Preparing Stage 3 Reports and Delisting Areas of Concern

Director, Great Lakes Division


[1] The federal government has provided GLAP funding since 1989 ($125 million over 5 years in 1989; $150 million over 6 years in 1994; $40 million over 5 years in 2000; and $40 million over 5 years in 2005). Identification, evaluation and remediation of AOCs have been a major element of GLAP funding since 1989 and the single focus of GLAP funding since 2000.

[2] Federal actions reference the Government of Canada's actions in response to its commitments under the COA (as opposed to Ontario's). These vary by AOC.

[3] Unbundling refers to the process of realigning temporary funding allocations (originally intended to deliver programs for various specific purposes) toward other departmental priorities.

Return to Table of Contents

1.0 Introduction

In the 2009-2010 fiscal year, Environment Canada's Audit and Evaluation Branch, Evaluation Division, commissioned an evaluation of the Great Lakes Action Plan IV (GLAP IV). This program was selected for evaluation prior to the end of the program's terms and conditions at the close of the 2009-2010 fiscal year.

This Final Report presents the findings of the evaluation. Chapter One provides background information on GLAP IV. Chapter Two discusses the objectives of the evaluation and the methodological approach. Chapter Three presents the evaluation's findings related to relevance, design and delivery, program outcomes, cost-efficiency and cost-effectiveness. Conclusions are included in Chapter Four and recommendations in Chapter Five.

1.1 Program Profile

The five Great Lakes – Superior, Michigan, Huron, Erie and Ontario – hold one fifth of the freshwater on the Earth's surface and 80 percent of the lake and river water in North America. Approximately 30 per cent of Canadians live within the Great Lakes Basin and 45 per cent of Canada's industry is located in this area. Ensuring environmental quality in this important region has implications for the natural environment, the health and well-being of Canadians, and Canada's economic competitiveness.

The overarching mechanism for protecting and restoring the Great Lakes is the Great Lakes Water Quality Agreement (GLWQA) (1972, 1978, 1987), which is a binational agreement between Canada and the United States. The Agreement sets out a series of commitments for both countries with respect to the Great Lakes. The International Joint Commission[4](IJC) assists the parties (Canada and the United States) in the implementation of the GLWQA. The most recent amendment to the Agreement was in 1987 and negotiations are under way for a renewed GLWQA. A key feature of the 1987 Agreement was the requirement that Canada and the United States take remedial action in heavily degraded locations or "areas of concern" (AOCs) around the lakes.[5] AOCs were identified in 1985 by the IJC and relevant federal, provincial and state governments. In total, 17 AOCs[6] were identified in Canada – twelve in Ontario and another five (along connecting channels) shared by Canada and the United States.

Canada's responsibility for managing and protecting the Great Lakes involves both federal and provincial (Ontario) jurisdictions. Both levels of government share authority to protect the environment, and are involved in aspects such as water, agriculture, species and spaces, and fisheries. Coordination of federal and provincial efforts with respect to the Great Lakes is accomplished through the Canada Ontario Agreement Respecting the Great Lakes Basin Ecosystem (COA). The goals and objectives of the COA parallel those of the GLWQA, and thus the Agreement includes an Annex on AOCs.

At the federal level, the Great Lakes Basin Ecosystem Initiative (GLBEI) is Environment Canada's mechanism for coordinating and delivering on federal commitments stemming from the GLWQA and the COA with respect to the Great Lakes Basin ecosystem. Remediation of domestic and binational Great Lakes AOCs is one of three priorities of the GLBEI.[7]

The Great Lakes Action Plan for Areas of Concern is the primary vehicle under the GLBEI for the federal government, with Environment Canada as the lead department, to act to restore AOCs around the Great Lakes area and fulfill Canadian commitments under the GLWQA. The current fourth phase[8] of the Action Plan, GLAP IV, was approved in 2005 with a budget of $40 million ($8 million per year over five years) to restore, protect and conserve AOCs around the Great Lakes.

Top of Page

1.1.1 Program Goals

The GLWQA states that "...the Parties, in cooperation with the State and Provincial Governments and the Commission, shall identify and work toward the elimination of ... Areas of Concern... Remedial Action Plans ... shall embody a systematic and comprehensive ecosystem approach to restoring and protecting beneficial uses in Areas of Concern... The Parties shall cooperate with State and Provincial Governments to ensure that Remedial Action Plans are developed and implemented for Areas of Concern."

The goals of GLAP IV are:

  • to make progress on federal[9] actions in eight AOCs: St. Mary's River, St. Clair River, Toronto and Region, Hamilton Harbour, Jackfish Bay, Detroit River, Niagara River and Port Hope; and
  • to complete federal actions in seven AOCs: Thunder Bay, Nipigon Bay, Peninsula Harbour, Bay of Quinte, Wheatley Harbour, St. Lawrence River and Spanish River.

To achieve these goals, GLAP IV is based on an ecosystem approach: remedial efforts targeted to AOCs under GLAP IV deal with interrelated environmental and sustainable development issues. As laid out in the GLWQA, remediation in each AOC is guided by Remedial Action Plans (RAPs). The original RAP reports were developed in 1987 by the federal and Ontario governments, with extensive public consultation. RAPteams with scientific/technical expertise for each AOC identified the nature and causes of environmental problems, and recommended actions, implementation plans and monitoring strategies. RAP reports are issued and updated for three stages of implementation: Stage 1 – Problem definition; Stage 2 – Selection of remedial measures; and Stage 3 – Restoration of beneficial uses.

The definition and characterization of the environmental problems in the AOCs is founded on identification of the beneficial uses that are impaired, the degree of impairment and the geographic extent of such impairment. Beneficial use is described as "the ability of living organisms to use the ecosystem without adverse consequence." There are 14 beneficial use impairments (BUIs) that are assessed:

  1. Restrictions on fish consumption
  2. Tainting of fish and wildlife flavour
  3. Degradation of fish and wildlife populations
  4. Fish tumours or other deformities
  5. Bird (or other animal) deformities or reproduction problems
  6. Degradation of benthos (organisms living in or near marine sediment environments)
  7. Restrictions on dredging activities
  8. Eutrophication or undesirable algae
  9. Restrictions on drinking water consumption, or taste and odour problems
  10. Beach closings / water contact‑sports restrictions
  11. Degradation of aesthetics
  12. Added costs to agriculture or industry
  13. Degradation of phytoplankton and zooplankton populations
  14. Loss of fish and wildlife habitat

Prior to the renewal of the GLAP in 2005, a total of 98 BUIs were identified across the 15 Canadian AOCs, and 14 additional beneficial uses required further evaluation. Using this information, RAP reports were updated and the priority work required in each AOC for delisting was identified. Removal of the designation as an AOC and monitoring of recovery indicates that the identified beneficial uses are no longer impaired and the area is restored. An Area in Recovery is a designation used to indicate that all remedial actions have been taken and the area must enter a period of natural recovery.

Top of Page

1.1.2 Program Activities

Upon renewal of the GLAP in 2005, Environment Canada, with federal partners, undertook a work planning and priority-setting effort. The federal partners that participated were the National Water Research Institute (NWRI) of Environment Canada, the former Environmental Conservation Branch (ECB),[10] the Department of Fisheries and Oceans (DFO), and Public Works and Government Services Canada (PWGSC).

The recommendations of the GLAP Work Plan Review Team were approved by the federal Great Lakes Executive Committee (GLEC) and signed off by the Environment Canada GLAP program director. GLAP IV funding was distributed to Environment Canada, DFO and PWGSC in April 2006.

Priority Investments

Three broad categories of priority investments under GLAP IV within AOCs are:

  • Remedial actions – Remedial actions within AOCs, consistent with the mandate of the Department, typically administered through the Great Lakes Sustainability Fund (GLSF):
    • contaminated sediment assessment activities, such as decisions on sediment management measures, and completion of contaminated sediment risk management strategies or engineering studies;
    • habitat restoration activities, such as supporting habitat creation projects, supporting stewardship and habitat acquisition, and habitat identification, restoration and protection;
    • support to municipal infrastructure improvements, including pre-engineering design studies for infrastructure upgrades, actions to secure partner funding and municipal commitment, and evaluation of cost-effective technologies for municipal wastewater treatment.
  • Science – Assessment and monitoring of BUIs:
    • assessments of ecosystem health based on the 14 BUIs identified in the GLWQA, to determine the need and effectiveness of remedial actions and eventually to confirm restoration of ecosystem health and therefore support official delisting of the AOC.
  • Engagement/Governance – Cooperative development and updating of RAPs with partners, consultation with the public:
    • management of federal programs in AOCs and coordination with Ontario and local governments, the private sector, non-governmental organizations (NGOs) and individuals engaged in the remediation of AOCs.

Excluding GLSF projects, approximately 40 federal science and monitoring projects were recommended for implementation over the life of the program. Table 1.1 displays these projects by AOC priority grouping and Table 1.2 by investment priority. This information is drawn from the recommended work plans for federal partners; information on actual project expenditures is not known, due to gaps in financial tracking of this information within Environment Canada (discussed in section 3.2.4).

Table 1.1: Federal Partner Projects Recommended for Funding: Priority AOC Grouping*
 Group 1 Priority AOCsGroup 2 Priority AOCsTotal**

Source: Environment Canada. GLAP IV Total Work Plan, August 21, 2008.xls. Internal document.

* The priority groupings are based on those identified by the internal Review Committee recommendations and not those identified in the GLAP IV program documentation. The only difference is the Review Committee group 1 AOCs includes the St. Clair River and excludes the Spanish River.

** Spanish Harbour (in recovery) and Port Hope were not identified as priority (group 1 or 2) AOCs, and so are not represented in Table 1.1. None of the GLAP IV projects led by other government departments (OGDs) and Environment Canada branches between 2005 and 2009 targeted these two AOCs. Environment Canada's Science and Technology Branch studied Spanish Harbour in 2005 and 2009, and the Department's Water Quality Monitoring and Surveillance group conducted water quality monitoring there in 2008.

*** Projects can address multiple AOCs falling into both of the priority groupings.

Environment Canada – ECB111122
Environment Canada – NWRI6511
PWGSC336
DFO81422
Total283361***

Table 1.2: Number and Value of Federal GLAP IV Projects Recommended for Funding by Priority Area and Partner

Great Lakes Sustainability Fund (GLSF)

A component of GLAP IV, the GLSF is a contribution funding mechanism to foster partnerships with other agencies and local community stakeholders to advance the goals of GLAP IV. Projects carried out by DFO and PWGSC were also funded through the GLSF.

The GLSF provides technical and financial support (up to one third of the total cost) to projects that implement remedial actions to support cleanup and restoration in four key priority areas: fish and wildlife habitat rehabilitation and stewardship, contaminated sediment assessment and remediation, innovative approaches to improve municipal wastewater effluent quality, and the elimination of non-point source pollution from agricultural sources.

GLSFfunding priority is given to those submissions undertaking remedial actions identified in the COA and in RAPs as federal government responsibilities, as well as science and monitoring activities essential to supporting the design and evaluation of these actions.[11] First priority for funding is given to those AOCs with the greatest potential for delisting in the short to medium term (group 1 AOCs as defined in the COA), and second priority is assigned to AOCs with less potential for delisting (group 2 AOCs).[12]

The GLSF uses a directed approach in soliciting project proposals: all proposals must link closely with intended goals and results listed in the COA, which in turn identify work required to delist AOCs. The proposal assessment criteria further include:

  • the need for federal participation through legislated mandate, existing federal policy or declared federal interest (the GLSFdoes not support funding for capital or operating costs of municipal infrastructure);
  • advances the completion of federal actions toward the recovery and delisting of AOCs;
  • the benefit that a project has towards specific measurable achievements;
  • environmental impact of a project (e.g., results of environmental assessments);
  • project's actual or potential technical merits;
  • application of the "polluter pays" principle, and two-thirds funding provided by other financial supporters;
  • input and endorsement by review agencies and peers, including RAP Implementation Teams where projects support remedial measures;
  • opportunities for technology transfer to other AOCs and the Great Lakes ecosystem;
  • project's ability to promote innovation and support new technologies that feed economic growth, create new opportunities and provide long-term improvements; and
  • achievement of environmental benefits at the lowest possible cost.

GLSFproposals are submitted to a multi-disciplinary review, which can include subject matter experts and AOC committee or RAP management team members, to provide external opinion on the proposed project in the form of a technical review of the proposed project and its alignment with regional priorities. A total of 267 GLSF contribution agreements worth nearly $16 million have been supported through GLAP IV since 2005, with the majority (n=158) targeting group 2 AOCs (those with less potential for delisting) – an activity focus that appears to be at odds with the stated intention of the program to focus on group 1 AOCs.

Table 1.3: Value and Number of GLSF Contribution Agreements, 2005 2010

Top of Page

1.1.3 Stakeholders and Recipients

The achievement of GLAP IV program objectives depends on the engagement of a variety of stakeholders. As indicated above, Environment Canada is the lead department for GLAP IV, and the program is the responsibility of the Strategic Integration and Partnerships Division in the Ontario Region. Scientific support for GLAP IV is provided by internal Environment Canada partners, particularly the NWRI, the Canadian Wildlife Service and Environmental Protection Operations Directorate within the Environmental Stewardship Branch, and the Water Science and Technology Directorate within the Science and Technology Branch. Although Environment Canada has primary responsibility for the delivery of GLAP IV, the program nonetheless relies on the technical and scientific expertise of partner OGDs (other government departments) to undertake key activities for the remediation of AOCs and to provide guidance (through their participation on various oversight committees, such as GLEC, etc.) on priorities for the remediation of these areas. Federal partners include DFO, Health Canada, Agriculture and Agri-Food Canada, Transport Canada, Parks Canada Agency, Natural Resources Canada and PWGSC.[13]

Other government jurisdictions are engaged through formal agreements: Canada and the United States are signatories to the GLWQA, and six federal departments and three provincial ministries are signatories to the COA.[14]

Partnerships are a requirement of GLSF projects. GLSFpartners include provincial and municipal governments, Conservation Authorities, NGOs, First Nations, educational institutions, local community volunteers, agriculture, industrial and other business sectors, and academia. These organizations may be proponents of GLSF-funded projects or act as partners in GLSF projects (such as in a funding, advisory or implementation role).

Finally, the development and implementation of RAPs involves various partners, similar in type to those involved in GLSF projects. Local implementation teams or councils typically include federal representatives (e.g., DFO, Environment Canada), other government representatives (Ontario Ministry of the Environment [MOE] or Ministry of Natural Resources [MNR], municipal/regional, First Nations), Conservation Authorities, community groups and NGOs.

Top of Page

1.1.4 Governance

The Great Lakes Areas of Concern Section, Strategic Integration and Partnerships Division within Environment Canada, is responsible for the management and coordination of GLAP IV.[15] While the program is managed in the Ontario Region under the Regional Director General, it is functionally accountable to the Department's Assistant Deputy Minister, Environmental Stewardship Branch. Further management oversight of GLAP IV is provided by the Great Lakes Environment Office, which is also within the Strategic Integration and Partnerships Division.

Several horizontal, federal/provincial and binational committees oversee and manage GLAP IV (described in more detail in Annex A under separate cover):

  • GLEC: ensures Canada's commitments under the GLWQA are met through the effective and efficient delivery of GLAP IV.
  • GLAPWorkplan Review Team: annual review of five year work plans submitted by federal departments, in consultation with the GLEC.
  • COA Management Committee: responsible for the oversight and overall administration of the COA, it helps to coordinate federal/provincial work carried out in AOCs (as per Annex 1 of the COA) and to provide direction and decision making for AOC work.
  • COA Annex Implementation Committee: coordinates development and implementation of the work planning process across multiple agencies.
  • Great Lakes Binational Executive Committee: coordinates binational programs and activities (including in five shared AOCs targeted by GLAP IV).
  • RAP committees: maintain working-level links to community and provincial stakeholders and ensure that environmental needs at the AOC level are addressed. Each RAPcommittee has a federal project lead who reports RAP activities directly to the COA Annex Implementation Committee.

Top of Page

1.1.5 Resources

Budget 2005 provided federal funding of $40 million ($8 million per year over five years) for GLAP IV. Table 1.4 describes the original allocation of these funds across fiscal years.

Table 1.4: Allocation of GLAP IV Funds ($000s)
 2005 062006 072007 082008 092009 10Total
1 Most full-time equivalent staff are situated in a Burlington, Ontario facility owned by Environment Canada.
Salary16596596596596593,295
Employee Benefit Plan132132132132132660
Operating7,2097,2097,2097,2097,20936,045
Total8,0008,0008,0008,0008,00040,000

Table 1.5 summarizes the allocation by organization. Funds were transferred from Environment Canada to relevant departments through interdepartmental settlements on the basis of approved work plans, while the work plans prepared for Environment Canada projects specified that GLAP IV resources would be allocated directly to these organizations. Information on actual GLAP IV expenditures for the remaining years of the program is not available due to gaps in financial reporting of expenditures at Environment Canada.

Table 1.5: Summary of Work Plans – Recommended Funding ($000s)
OrganizationAnnual Recommended FundingTotal Recommended Funding 2005 2010
DFO5732,866
Environment Canada – ECB1,1939,568
Environment Canada – GLSF3,78418,918
Environment Canada – NWRI1,7308,648
Total8,00040,000

Top of Page

1.1.6 Program Logic Model

A logic model is a visual representation of a program/initiative that identifies the linkages between an initiative's activities and the achievement of its outcomes. Figure 1 shows the program logic model, which presents a graphical depiction of how the activities and outputs of GLAP IV relate to immediate, intermediate, long-term and ultimate outcomes.

Figure 1: Logic Model – GLAP IV

This figure shows the program logic model, which presents a graphical depiction of how the activities and outputs of the Great Lakes Action Plan IV relate to immediate, intermediate, long-term and ultimate outcomes.
Click to enlarge

Top of Page

1.1.7 Performance Reporting and Evaluations

There are a number of reporting exercises related to federal Great Lakes ecosystems programs:

  • GLWQA: The parties to the agreement (Canada and the United States) report biennially to the IJC on progress in developing and implementing RAPs. The Commission reports to federal, state and provincial governments every two years on progress in relation to achievement of GLWQA goals and objectives, and provides ongoing assessment and advice to governments through semi-annual meetings and special reports. RAPs for each AOC are also submitted to the Commission for review and comment at three stages:
    • Stage 1 RAPs – upon completion of definition of the problem and definition of the causes of the use impairment (including a description of all involved known sources of pollutants and an evaluation of other possible sources);
    • Stage 2 RAPs – submitted when remedial and regulatory measures are selected, and are to include a schedule for their implementation, as well as identification of the persons or agencies responsible for remedial measures implementation; and
    • Stage 3 RAPs – submitted when monitoring indicates that identified beneficial uses have been restored.
  • COA: A multi-year work plan is developed and updated annually for the COA, based on input from all the participating federal departments and provincial ministries that are parties to the COA. Reporting on activities and achievements in the work plans is analyzed and evaluated in order to ensure that key contributions are being made by participating agencies to enable achievement of COA commitments.

While the GLAP programs themselves have not been the subject of a program evaluation, other performance reporting initiatives have included:

  • a mid-term review of GLAP IV;
  • a review of the GLBEI by the Commissioner of the Environment and Sustainable Development (CESD) completed in 2008 (a follow-up to the 2001 CESD audit); and
  • participation of the GLBEI in a Treasury Board Management Accountability Framework (TB MAF) exercise.

[4] The IJC was established in 1909 by the Boundary Waters Treaty between the United States and Canada. The role of the IJC is described in detail in Article VII of the GLWQA.

[5] The 1987 amendments also mandated the development and implementation of lakewide management plans, and committed both countries to control pollution from non-point sources, identify the nature and extent of sediment pollution, and develop methods to evaluate the impact of contaminated sediments and the technological capabilities of programs to clean them up.

[6] Two of these AOCs have since been delisted: Collingwood Harbour in 1994 and Severn Sound in 2003. Although officially announced on April 16, 2010, the delisting of Wheatley Harbour was not factored into the analysis of outcomes achievement, as it occurred outside the time frame for GLAP IV, which ended on March 31 2010, and subsequent to data collection and analysis for the present evaluation.

[7] Development and implementation of binational lake-wide management plans for the restoration, conservation and protection of the ecosystems of each of the five Great Lakes and Lake St. Clair, and implementation of the Great Lakes Binational Toxics Strategy (which deals with the development and implementation of challenge goals for the reduction of priority persistent toxic substances within the Great Lakes Basin), are the other two priorities of the GLBEI (Treasury Board Secretariat 2008 Strategic Review of GLBEI).

[8] The federal government has provided GLAP funding since 1989 ($125 million over 5 years in 1989; $150 million over 6 years in 1994; $40 million over 5 years in 2000; and $40 million over 5 years in 2005). Identification, evaluation and remediation of AOCs have been a major element of GLAP funding since 1989 and the single focus of GLAP funding since 2000.

[9] Federal actions reference the Government of Canada's actions in response to its commitments under the COA (as opposed to Ontario's). These vary by AOC.

[10] When it existed, the ECB included a number of divisions, four of which received GLAP IV funding: EHD (Ecosystem Health Division), RPD (Restoration Programs Division), GLSFD (Great Lakes Sustainability Fund Division), and CSD (Conservation Strategies Division). During departmental transformation, the divisions in the former ECB were restructured as follows: EHD went to Science and Technology Branch; part of CSD went to the Wildlife and Landscape Science Division within the Science and Technology Branch, while the other part of CSD became the Canadian Wildlife Service – Ontario Region (CWS‑ON) within the Environmental Stewardship Branch; and both RPD and GLSFD went to the Regional Director General's Office, and subsequently were restructured into the Great Lakes Management and Reporting Section and the Great Lakes Areas of Concern Section.

[11] The COA assigns federal or provincial leadership for each AOC. According to the COA Annex: a) Canada and Ontario will co-lead the RAP process in the Toronto and Region, St. Mary's River, St. Clair River, and Detroit River AOCs; b) Canada will lead the RAP process in the Thunder Bay, Hamilton Harbour, Port Hope and St. Lawrence River AOCs; and c) Ontario will lead the RAP process in the Nipigon Bay, Jackfish Bay, Peninsula Harbour, Spanish Harbour, Wheatley Harbour, Niagara River and Bay of Quinte AOCs.

[12] In the 2007 2010 Canada Ontario Agreement, Group 1 AOCs are Nipigon Bay, Jackfish Bay, Wheatley Harbour and St. Lawrence River (Cornwall).

[13] Some of these departments were invited to submit proposals for GLAP IV but declined, and were therefore not represented on the internal Review Committee. However, all OGDs listed here are members of the GLEC.

[14] Federal partners include Environment Canada, Parks Canada Agency (not a separate signatory, as the EC Minister is also responsible for Parks Canada), Agriculture and Agri-Food Canada, Fisheries and Oceans Canada, Health Canada, Natural Resources Canada, and Transport Canada.  Provincial signatories to the agreement include the of the Environment, the Ministry of Natural Resources and the Ministry of Agriculture, Food and Rural Affairs.

[15] The Areas of Concern Section, Great Lakes Environment Office, and Great Lakes Management and Reporting Section now report through the Great Lakes Division.

Return to Table of Contents

2.0 Evaluation Design

2.1 Objectives and Scope

The evaluation covers GLAP IV program activities from the 2005‑2006 to 2009‑2010 fiscal years. The evaluation findings will be used to assist the Great Lakes AOC program and other departmental stakeholders in the ongoing management of GLAP IV, and will contribute to efforts (beginning in February 2010) related to the renewal of the program for another five-year period. The evaluation issues include:

  1. Is there a continued need for a GLAP?
  2. Is the GLAP IV aligned with federal government priorities?
  3. Is the GLAP IV consistent with federal roles and responsibilities?
  4. To what extent have intended outcomes been achieved as a result of GLAP IV?
  5. Is the GLAP IV design appropriate for achieving expected program results?
  6. Have there been any unintended (positive or negative) outcomes?
  7. Is GLAP IV undertaking activities and delivering products in the most efficient manner?
    • How could the efficiency of the program's activities be improved?
    • Are there alternative, more efficient ways of achieving the program's objectives?
  8. Is GLAP IV achieving its intended outcomes in the most economical manner?

The complete matrix of evaluation questions, indicators and data sources is presented in Annex B (under separate cover).

2.2 Approach and Methodology

Multiple lines of evidence were used to increase the reliability and robustness of the analysis. The evaluation included a review of documentation and literature, a review of GLSF and federal GLAP IV project files, and key informant interviews. Data collection occurred between February 3 and March 5, 2010.

2.2.1 Document and Literature Review

Secondary documentary sources were reviewed to develop a detailed program profile and contribute to addressing several of the evaluation questions, including relevance and program performance (achievement of program outcomes and the cost-efficiency analysis in particular). Environment Canada provided documents such as: background/planning and scoping materials; performance documents (e.g., Evaluation Plan for the Great Lakes Action Plan, Mid-term Review of GLAP IV); corporate/policy documents; reports on the Great Lakes (e.g., IJC reports); program work plans / progress reports; and other material. A document review template was developed to summarize findings in the documents pertaining to the evaluation questions. A bibliography of key documents is included in Annex C (under separate cover).

Top of Page

2.2.2 File Review

A review was conducted of a sample comprising 39 of a total of 234 files for contribution projects funded between 2005‑06 and 2008‑09 by the GLSF program component.[16] The sampling strategy was devised by Environment Canada, taking into consideration fiscal year, AOC priority group, and project value strata[17] so that the sample reflected the population. Annex D (under separate cover) provides a profile of the sample and population of GLSF files.

A data collection template was used to capture file information in a consistent manner, and to ensure that the content of the files was well-documented and linked to the specific evaluation questions and indicators (see Annex E under separate cover). The file review addressed issues related to the consistency of contributions to departmental objectives, the achievement of outputs and outcomes, unintended impacts, and project costs and leveraging. A profile of the characteristics of the files that were reviewed is included in Annex E (under separate cover).

The GLSF files were generally well documented, though there were some gaps. Contents typically included the project proposal (92 per cent of files), memorandum of understanding (MOU) or contribution agreement (90 per cent of files), financial information or records (e.g., invoices; 77 percent of files) and proposal review / technical review information (69 per cent of files). Almost 80 per cent of files had some output or outcome reporting (i.e., an annual report, progress report or final report). It is not clear why some files do not have output or outcome reporting. Table 2.1 indicates the extent to which key documents were included in the files that were reviewed.

Table 2.1: Distribution of Documents in GLSF Files Reviewed
Type of documentPercent of files
Proposal92%
MOU / Contribution Agreement90%
Financial files/records77%
Proposal review/approvals69%
Final report62%
Correspondence56%
Technical screening31%
Progress reporting / activity reporting28%
Annual report18%
Meeting/committee agendas/minutes8%
Audit/evaluation reporting5%
Other products26%
Other (e.g., brochures, media releases, special technical reviews, photographs, CDs)49%

Files for 21 projects sampled from those conducted by federal partners were also requested. All funded partners, including DFO, the former ECB, NWRI and PWGSC provided project-related documentation. However, the documentation was in various formats (e.g., conference presentations, reports, scientific publications, invoices/contribution agreements) and reporting was not always linked to a single project, thus was not amenable for capture using a standardized file review methodology. As a result, descriptive analyses were undertaken to summarize the contents of the federal files (see Annex F under separate cover). To better understand the reporting requirements and project-level documentation for federal projects, the key informant interview guides for federal partners and federal project proponents were expanded to address this issue.

Top of Page

2.2.3 Key Informant Interviews

In total, 46 key informant interviews were completed with Environment Canada personnel, committee members, other government representatives, project proponents (funded and unfunded) and experts. The targeted distribution of interviews was met in all cases and is as follows:

  • Departmental program managers and federal partners involved in the management and delivery of GLAP IV (n=11);
  • Representatives from federal OGDs and Ontario who are members of committees (GLEC, GLAP Workplan Review Team, COA Management Committee, and COA Annex Implementation Committee (n=10);
  • Federal project proponents (n=4);
  • GLSF project proponents (n=10);
  • Unsuccessful applicants (n=4);
  • RAP committee members (n=4); and
  • International or academic experts on aquatic ecosystem remediation (n=3).

The interviews were conducted by phone and the duration of the interviews varied by respondent group, ranging from 30 to 90 minutes. Key informants were asked their views and opinions on a number of issues, including the continued need and relevance of the program, its design and delivery, the extent to which the program is meeting immediate, intermediate and longer-term outcomes, and the efficiency and economy of the program in achieving outputs and results. A master key informant interview guide is included in Annex G (under separate cover), from which individual interview guides were further tailored to be suitable for the various respondent groups.

To ensure a common understanding of the terms used in the analysis and reporting of interview results, the following guidelines have been used:

  • "A few/a small number of interviewees" = less than 25 per cent
  • "Some/a minority of interviewees" = 25‑49 per cent
  • "A majority of interviewees" = 50‑75 per cent
  • "Most interviewees" = over 75 per cent
  • "Almost all interviewees" = 95 per cent or more

Top of Page

2.3 Challenges and Limitations

A number of methodological caveats should be noted for this evaluation. First, much of the information that was gathered for this review is drawn from internal sources: program documentation as well as interviews with program managers or federal partners, committee members and project proponents. With respect to program documentation, the main disadvantage is that these secondary sources often reflect stated intentions of a program (not implementation) and are not necessarily produced for the explicit purposes of the evaluation, thus are often not organized to present, or do not touch upon, issues related to those investigated through an evaluation. With respect to the interviews, respondents frequently had some prior connection to the program (e.g., unfunded applicants who were funded for another project, or experts who worked at Environment Canada in the past). Although a small number of interviews were conducted with unfunded GLSF applicants and experts, and priority was given to interviewing key informants who had not been awarded GLSF funding, the interview feedback benefits from respondents' knowledge of the program but lacks a high level of objectivity.

Second, the availability of program activity information and financial data with respect to the GLAP IV is limited. Specifically:

  • Due to the departmental transformation in 2005‑2006 and unbundling of resources and activities of the GLBEI,[18] it is not possible to track Environment Canada departmental GLAP IV expenditures. Expenditures were being tracked at the ecosystem initiative level and not coded at the program level. This gap limits the cost-efficiency analysis conducted for the evaluation.
  • The GLSF program database is out-of-date with respect to entries after 2006‑2007, and key measures such as financial expenditures and leveraged funding from partners and program activities are either not captured or not captured systematically year-to-year. This was due to a shift from the use of MOUs to a grants and contributions (G&C) funding mechanism (with different reporting templates and a centralized information management system), as well as the internal re-organization of the unit.
  • There is no centralized database for performance measurement purposes for the program.

As a result, it was not possible to provide a complete picture of program activities and outputs. As well, the cost-efficiency analysis for the program was limited by the lack of clear financial information, though this analysis was supplemented by qualitative assessments of cost-efficiency obtained in the key informant interviews.

Finally, it is important to note that the determination of progress in AOCs and the achievement of program outcomes, particularly related to restoration and delisting, are only realizable in the longer term. The five-year term of GLAP IV and, indeed, the 20 years of GLAP funding are widely perceived to be insufficient to address the complex ecological issues in the AOCs. In addition, the achievement of longer-term GLAP IV program outcomes (e.g., delisting) requires the significant contribution of other partners and stakeholders. External factors, such as the availability of funding for infrastructure projects, are critical to the achievement of program outcomes. Thus, evaluation findings related to longer‑term outcomes of GLAP IV must be considered in this context.

Top of Page

2.4 Reporting Note

In the next chapter, findings are presented for the following five evaluation issues explored through the evaluation: relevance, design and delivery, achievement of program outcomes, efficiency, and economy.[19] A rating is also provided for each evaluation question. The ratings are based on a judgement of whether the findings indicate that:

  • the intended outcomes or goals have been achieved or met (labelled as Achieved);
  • considerable progress has been made to meet the intended outcomes or goals, but attention is still needed (labelled as Progress Made, Attention Needed);
  • little progress has been made to meet the intended outcome and attention is needed on a priority basis (labelled as Little Progress, Priority for Attention); and
  • a rating is not applicable (identified by the N/A symbol).

In addition, a tilde symbol (~) is used to denote instances where outcome achievement ratings are based solely on subjective evidence. A summary of ratings for the evaluation issues and questions is presented in Annex H (under separate cover).


[16] GLSF files from 2009‑2010 were not sampled due to the recency of their approval and the limited reporting available for these projects during the evaluation data collection period.

[17] A homogeneous subgroup of members of the population.

[18] The process of transformation was designed to enable the Department to plan, manage and report by results. Transformation involved re-defining the results structure (Program Activity Architecture) and new management structures and processes to promote integrated management and decision making in the context of a clearer view of departmental results and strategic direction (Environment Canada, Report on Plans and Priorities, 2006‑2007).

[19] Treasury Board policy requires that all evaluations of federal programs and initiatives address the core issues of relevance and performance. Performance is a blended construct that includes a program's effectiveness (achievement of outcomes), efficiency (achievement of outputs/activities at the lowest cost) and economy (achievement of outcomes at the lowest cost).

Return to Table of Contents

3.0 Findings

3.0 Findings

3.1 Relevance

This section presents the evaluation findings on the continued need for GLAP IV, the alignment of the program with federal and departmental priorities, and the extent to which GLAP IV is consistent with federal roles and responsibilities.

Overall Findings

The GLAP remains a relevant program that enables the federal government to address the continued need for restoration and maintenance of the Great Lakes AOCs. While recent trends in Great Lakes' ecosystem conditions are variable, historical sources of stress and new challenges are generating negative impacts in many areas of the lakes. Environmental need, together with the social and economic benefits of the Great Lakes, supports the continued relevance of the program. Public opinion is consistent with scientific information and key informant views on the environmental and societal importance of the Great Lakes and the continued need for the GLAP.

There is little redundancy risk associated with GLAP IV. The program uniquely addresses federal commitments in the GLWQA and the COA by providing a framework to address BUIs linked to AOC RAP priority areas. Unlike other public funding (federal, provincial and municipal) and non-public funding (industry, foundations, non‑governmental organizations, etc.), GLAP IV is targeted exclusively to AOCs. Key informants express few concerns about the possibility of duplication or overlap between GLAP IV and other programming. Rather, GLAP IV is seen as highly complementary and a catalyst in orienting other funding sources to provide support in the AOCs.

GLAP IV is consistent with federal and departmental priorities. The program is part of the GLBEI, one of six ecosystem initiatives at Environment Canada under the Priority Ecosystem Initiatives Sub-Activity. The GLBEI supports the Department's strategic outcome that "Canada's natural capital is restored, conserved and enhanced." GLAP IV outcomes and priority investments also support a number of other sub-activities within Environment Canada's program activity architecture, including Aquatic Ecosystems, Wildlife, Priority Ecosystems, Ecosystems Sustainability, and Assessment and Ecological Monitoring.

Environment Canada's GLAP IV is aligned to support federal government commitments and obligations under the GLWQA and the COA, as well as federal government priorities outlined in the 2007 Speech from the Throne and two recent federal budgets. Other legislative authorities include the Canada Water Act, the International Boundary Waters Treaty Act, and the Canadian Environmental Protection Act, 1999.

Top of Page

3.1.1 Continued Need

  • Evaluation Issue: Is there a continued need for a GLAP?
  • Indicator(s):
    • Demonstration of societal/environmental need
    • Presence/absence of other programs that complement or duplicate the objectives of the program
    • Reach and activities are connected to societal/environmental needs
  • Methods:
    • Document review
    • File review
    • Key informant interviews
  • Rating: Achieved
Connection to Environmental/Societal Need

For over four decades, the GLWQA and COA have served as important and effective mechanisms for protecting and restoring the Great Lakes. Canadian and United States actions implemented under the GLWQA have resulted in the lowering of phosphorus loads to the Great Lakes, and improvements in several water quality indicators, in particular in the more heavily nutrient‑impacted lower lakes, as indicated in the State of the Lakes Ecosystem Conference's State of the Great Lakes 2009 report. At the same time, a 2007 review of the GLWQA by the IJC found that stronger linkages are needed between the Agreement's stated overall purpose – to restore and maintain the chemical, physical and biological integrity of the waters of the Great Lakes Basin ecosystem – and the measures outlined in the Agreement's articles and annexes. The review further emphasized that the Agreement should be revised to address today's pressing issues, including the impacts of climate change, aquatic invasive species and urbanization. There was also recognition that these issues affect biodiversity. Accordingly, in June 2009 the federal governments of Canada and the United States announced a commitment to strengthen and modernize the Agreement so as to better address pollution, invasive species and climate change.[20]

Recent trends of Great Lakes ecosystem conditions vary: some conditions are improving and some are deteriorating.[21] Efforts under the GLWQA directed at degradation and contamination in AOCs through the implementation of RAPs and lake‑wide management plans have coordinated activities directed at addressing numerous BUIs. Yet, despite the progress made to date, the Great Lakes are exhibiting symptoms of extreme stress from a combination of sources, including toxic contaminants, invasive species, nutrient loading, shoreline and upland land use changes, and hydrologic modifications. In large areas of the lakes, historical sources of stress have combined with new ones to reach tipping points, at which ecosystem-level changes occur rapidly and unexpectedly, confounding traditional relationships between sources of stress and expected ecosystem responses. In a 2008 TB MAF review of the GLBEI, an examination of the research concluded that, despite significant progress and many successes in addressing past challenges, the Great Lakes continue to be at risk of unprecedented changes due to a range of stresses acting in combination, including: degradation of the nearshore zone; continued introduction and spread of aquatic invasive species; significant changes to the lower food web; wetland and other natural habitat loss; chemical contamination; threats to drinking water; and impacts of climate change.

Among key informants, there is consensus across all respondent groups on the continued need for GLAP IV. Almost all interviewees agree that there is a need for federal funding to support the restoration and maintenance of Great Lakes AOCs. The Great Lakes are a priority ecosystem with strong economic and social importance for all Canadians. For most interviewees, the restoration and maintenance of the Great Lakes represents not only a shared responsibility among all jurisdictions (federal, provincial and municipal), but one for which federal leadership and funding is critical for a number of reasons (among them, the federal mandate for transboundary waters, in accordance with the Canada Water Act and the International Boundary Waters Treaty Act).

A majority of interviewed key informants indicate that there is a continued need for federal funding to ensure the coordination and leveraging of funding with other jurisdictions and stakeholders to restore and maintain the Great Lakes. A minority of interviewees (though a majority of committee members and external experts) refer to the federal government's jurisdictional responsibilities for international waters as well as those outlined in international and national agreements, as the rationale for the continued need for GLAP IV. A number of interviewees (including a majority of committee members and of federal managers and partners) cite the magnitude of the remaining issues and problems facing the Great Lakes in general, and the AOCs in particular, as evidence for the continued need for federal leadership and funding. For a few GLSF project proponents, the issue of "legacy sites" and non-point source contaminants, and the broader economic benefits of the lakes as a resource, represent additional reasons for the continued need for federal funding in support of the Great Lakes AOCs.

While the AOC designation is useful in focusing attention and resources on areas of stress, some key informant interviewees indicate there is also a need for a more holistic lake- or ecosystem-wide approach to adequately address environmental issues affecting the health of the Great Lakes ecosystems. Some respondents believe there is a need to expand the focus of GLAP IV beyond the designated AOCs. A few respondents indicate that the GLAP should be broadened to include activities identified in the lake-wide management plans developed for each of the Great Lakes. For a small number of interviewees, there is a perceived need to broaden the focus to include shoreline and watershed remediation activities considered to be important for the long‑term integrity and sustainability of the Great Lakes.

Most key informants believe all GLAP IV priority investment areas are important and that there is an ongoing need to continue supporting all targeted priority areas.[22] A majority of interviewees underscore the need for Environment Canada to be flexible in maintaining the GLAP's responsiveness to the different needs of the individual AOCs (i.e., that priorities need to be assessed on an AOC by AOC basis).

Public opinion is consistent with scientific and key informant views on the importance of the Great Lakes and the continued need for support. Recent public opinion research reveals a growing concern among Canadians about Canada's freshwater resources and the need for governments to ensure the availability of freshwater supplies. A 2009 Nanos poll found that six in ten Canadians (62 percent) consider freshwater to be our most important natural resource, compared to two out of ten (22 percent) who feel oil and gas are the most important. Also, four in ten Canadians (40 per cent) indicate that water pollution from industry, agriculture and/or households is their "greatest concern" regarding freshwater in Canada.

Canadians' confidence in the supply and safety of our freshwater resources has declined significantly in recent years. The 2009 Water Attitudes Study found that, while nearly all Canadians believe it is important to conserve freshwater on an ongoing basis (95 percent), confidence that Canada has enough freshwater for the long term has declined by 11 percentage points, from 81 per cent in 2008 to 70 per cent in 2009. Similarly, Canadians' confidence in the safety of Canada's freshwater supply has declined, from 81 per cent in 2008 to 72 per cent in 2009. Seven in ten Canadians (70 per cent) believe that government is the "most responsible" for ensuring the availability of clean water for their communities.[23]

The Great Lakes United Poll (2007) found that Ontarians are "very concerned" about untreated sewage entering the Great Lakes (82 per cent), contamination of the food chain by toxic pollution such as mercury (78 per cent), and the loss of wetlands and animal habitat (67 per cent). The poll also revealed that eight in ten Ontarians (78 per cent) favour spending $2 billion per year for ten years to restore the health of the Great Lakes and St. Lawrence River. Majorities of Ontarians agree that the following constitute "very good" reasons for spending Canadian tax dollars to clean up the Great Lakes: 10 million Canadians get their drinking water from the Great Lakes (83 per cent of Ontarians consider this a "very good" reason); we should act now because problems in the lakes will cost more to fix in the future (77 per cent); the Great Lakes represent one fifth of the world's freshwater supply (77 per cent); and contamination of the food chain (72 per cent). Ontarians also perceive a number of benefits to protecting the integrity of the Great Lakes, as evidenced by majorities of Ontarians who "agree" that: we all have a personal responsibility to leave the Great Lakes in good health for our children (99 per cent); government spending to upgrade sewage and water systems in the province will create jobs (89 per cent); and the Great Lakes will help shield us from the impacts of global warming like heat and drought (72 per cent).

Potential for Duplication and Overlap

Program documents and key informants point to the availability of several other funding programs that support remediation or science and monitoring in the Great Lakes AOCs. A key source is provincial funding for implementation of Ontario commitments under the COA. Relevant provincial ministries, the MNR and MOE, use these funds to fulfill provincial commitments in the COA, including those in Annex 1 pertaining to AOCs. In addition to provincial COA funds, funding sources include:

  • Environment Canada A-base funds (salary dollars for permanent staff involved in science and monitoring in the AOCs and elsewhere);
  • other Environment Canada funding programs (e.g., Habitat Stewardship Program, EcoAction Community Funding Program, Environmental Damages Fund) that may fund projects in the Great Lakes AOCs;
  • the Federal Action Plan on Clean Water, which provided $48.9 million for sediment remediation in the Great Lakes AOCs;
  • municipal funding;
  • foundations (e.g., the Trillium Foundation), associations (e.g., Ontario Federation of Anglers and Hunters), NGOs (e.g., Ducks Unlimited), universities;
  • Infrastructure Canada funding for infrastructure projects such as wastewater treatment improvements; and
  • industry-provided funding.

Unlike other funding sources, however, GLAP IV is uniquely targeted to remediation work in the Great Lakes AOCs. Also distinguishing GLAP IV from other initiatives is the program's framework for funding that is closely tied to Canada's stated AOC commitments under the GLWQA, priorities identified in the RAP reports in each AOC, and its science-based focus. Funding and work planning is thus linked to BUIs identified in each AOC, with the objective of completing federal actions to address BUIs in order to delist.

Key informants across all respondent groupings expressed few concerns about duplication or overlap between GLAP IV and other funding sources. Mitigating the risk of duplication between GLSF-funded projects and other funding sources is informal information-sharing at the officer level – for example, sharing of GLSF project proposal and funding lists among officers with responsibilities for the various funding programs. Also, more formally, the technical review of project proposals may also bring to light potential areas of funding overlap or areas where efficiencies among projects can be achieved. While GLSF project proponents also perceived GLSF and other funding sources to be complementary rather than overlapping, they noted that there was duplication of effort for project proponents in terms of their efforts to access funds from multiple sources (i.e., preparing funding applications for various funding programs).

The risk of duplication with respect to funding of science and monitoring activities is mitigated because work in the AOCs is conducted by a relatively small science community, with effective collaboration and awareness of each other's activities between federal researchers and their provincial and U.S. counterparts. As well, each AOC has an assigned federal or provincial lead, which further ensures that duplication and overlap is avoided.[24]

Summary: Recent trends of Great Lakes ecosystem conditions vary; some conditions are improving and some are deteriorating. In large areas of the lakes, historical sources of stress have combined with new ones to reach a tipping point, at which radical ecosystem-level changes can occur rapidly and unexpectedly. Public opinion is consistent with scientific and key informant views on the environmental and societal importance of the Great Lakes and the continued need for the GLAP. GLAP IV is targeted exclusively to AOCs, unlike other public (federal, provincial and municipal) and non-public (industry, foundations, NGOs, etc.) initiatives. Key informants express few concerns about the possibility of duplication or overlap between GLAP IV and other programming. Rather, GLAP IV is seen as complementary in that it is a catalyst for orienting other funding sources to support activities in the AOCs.

Top of Page

3.1.2 Alignment with Federal and Departmental Priorities

  • Evaluation Issue: Is GLAP IV aligned with federal government priorities?
  • Indicator(s):
    • Program's objectives correspond to recent/current federal government priorities
    • Program's objectives are aligned with current departmental strategic outcomes
    • Views on the alignment of program objectives with current federal government and departmental priorities
  • Methods:
    • Document review
    • Key informant interviews
  • Rating: Achieved
Federal Priorities

Environment Canada's GLAP is aligned to support federal government priorities established under the 2007 Speech from the Throne and two recent federal budgets. Notably, one of the five priorities set out in the October 2007 Speech from the Throne is "A Healthy Environment for Canadians." The Throne Speech commits the Government of Canada to implementing a new "water strategy" to help clean up major lakes and oceans. Budget 2007 created the Action Plan for Clean Water, which allocated funds to improve the quality of water in Canada's rivers, lakes and oceans, including $48.9 million over eight years to accelerate existing actions for the remediation of contaminated sediment in eight Great Lakes AOCs.

The Great Lakes are among the high-priority ecosystems identified nationally and recommended for coordinated action.[25] This recommendation was identified through the 2007‑2008 Priority Ecosystem Initiatives Management Framework initiative, which was intended to improve the Priority Ecosystems Initiatives program, optimize integration of Environment Canada programs and activities, and strengthen accountability and reporting. Ongoing federal support is further evident in Canada's signing in 2007‑2008 of a new three-year COA to work toward a healthy and clean Great Lakes Basin Ecosystem.[26]

Almost all key informants concurred that the objectives of GLAP IV are well-aligned with current federal government priorities, pointing most often to the GLWQA. By signing this Agreement, Canada has an obligation to work toward improving the Great Lakes. Some interviewees further point to the federal interest in water quality (relating to the 2007 Throne Speech, and Budget 2007 establishment of an Action Plan for Clean Water and a priority on providing "A Healthy Environment for Canadians"[27]) as evidence of GLAP IV alignment with federal priorities. In addition, some key informants (both internal departmental managers and partners, and external experts and RAP representatives) note that GLAP IV is an important vehicle to fulfill international commitments and to contribute to positive Canada‑U.S. relations.

Departmental Priorities

The GLBEI is one of six ecosystem initiatives at Environment Canada under the Priority Ecosystem Initiatives Sub-Activity, and aligns with one of the department's seven program priorities (Ensuring Water Quality and Quantity).[28] The Priority Ecosystem Initiatives Sub‑Activity adopts an ecosystem approach to environmental management for the benefit of governments, industry and individuals, by aligning science, monitoring, on‑the‑ground action and policy expertise, as well as enhancing collaborative governance and decision-making mechanisms. The GLBEI supports the Department's strategic outcome that "Canada's natural capital is restored, conserved, and enhanced." GLAP IV outcomes and priority investments were designed to complement a number of other sub-activities within Environment Canada's program activity architecture, including Aquatic Ecosystems, Wildlife, and Assessment and Ecological Monitoring.

GLAP IV is considered by most key informants to be well-aligned with Environment Canada departmental priorities as a priority ecosystem. A number of federal respondents also note that activities within the Great Lakes, such as science and monitoring, have national applicability and that this research can be used to benefit areas beyond the Great Lakes. Furthermore, GLAP IV is viewed by key informants as a consistent fit with Environment Canada's mandate to preserve and enhance the quality of the natural environment (habitat, biodiversity, species at risk) and conserve and protect Canada's water resources.

Summary: GLAP IV aligns well with federal and departmental priorities. The program uniquely addresses federal commitments in AOCs under the GLWQA and COA, by providing a framework to address BUIs linked to AOC RAP priority areas. The GLBEI, of which GLAP IV is a part, is one of six ecosystem initiatives at Environment Canada under the Priority Ecosystem Initiatives Sub-Activity. The GLBEI supports the Department's strategic outcome that "Canada's natural capital is restored, conserved, and enhanced." GLAP IV outcomes and priority investments support a number of sub‑activities within Environment Canada's program activity architect, including Aquatic Ecosystems, Wildlife, and Assessment and Ecological Monitoring.

Consistency with Federal Roles and Responsibilities
  • Evaluation Issue: Is GLAP IV consistent with federal roles and responsibilities?
  • Indicator(s):
    • Program mandate aligned with federal government jurisdiction
    • Views on the appropriateness of federal involvement
  • Methods:
    • Document review
    • Key informant interviews
  • Rating: Achieved

The Government of Canada, through the GLWQA, has committed to working with other levels of government to assess, restore and protect beneficial uses in AOCs. Legislative authorities include:

  • The Canada Water Act, which establishes and reinforces Environment Canada's mandate in the management and protection of water quality in Canada. This Act authorizes the Minister of the Environment to enter into agreements with provincial governments, subject to Governor in Council approval, where there is a significant national interest in the management of a water resource, and to work with provinces in designing and implementing projects for the efficient conservation, development and utilization of those waters. Authority is also granted to establish joint commissions, boards or other bodies empowered to direct, supervise and coordinate those programs.
  • The International Boundary Waters Treaty Act, which provides the principles and mechanisms to help resolve disputes and to prevent future ones, primarily those concerning water quantity and quality along the boundary between Canada and the United States. This Act supports the establishment of the IJC under the boundary waters treaty signed by Canada and the United States in 1909.
  • The Canadian Environmental Protection Act, 1999, which is the federal legislation respecting pollution prevention and the protection of the environment and human health, in order to contribute to sustainable development.

Key informants across all respondent groups concur that it is appropriate for the federal government to play a role in the restoration and maintenance of the Great Lakes AOCs. Collectively, key informants point to several factors that illustrate the need for a federal role in the Great Lakes. First, the federal government has a responsibility to support work to fulfill international agreements (such as the Great Lakes Water Quality Agreement). Further, as the Great Lakes are the largest freshwater body in North America, it is considered by the majority of key informants to be a critical national resource. For example, one key informant stated that "they are clearly important to water quality, industry and transportation, and affect a significant proportion of the Canadian population."

The complexity of the boundaries of the waters – four of five lakes form part of the Canada-U.S. border and five AOCs are binational – underscores the appropriateness of a federal lead in this area. An external expert summarized this point by noting that "because we are dealing with binational issues, federal funding that enables place-based remediation and protection to happen is paramount to the success of the government's promise to the [Great Lakes Water Quality] Agreement."

Some key informants feel that fulfilling this commitment is especially important now that the Government of the United States has committed $475 million in new funding to the Great Lakes under the U.S. Great Lakes Restoration Initiative.[29] This increase in funding is perceived by some key informants to raise expectations for Canada to provide a commensurate investment in the Great Lakes. Through GLAP IV efforts, some key informants point out that Canada has taken a leadership role relative to the United States on Great Lakes AOC restoration and maintenance. They urged that these activities, along with the financial commitment, continue.

Although key informants agree that there is a need for federal involvement, including a leadership role, it was also noted that the federal government should not and cannot play the exclusive role, and that provincial and municipal governments, as well as community-driven organizations, have a responsibility and critical role to play as well. An example of this is that the jurisdiction for wastewater infrastructure improvements rests with municipalities and the provinces.

Summary: The federal role in GLAP IV is appropriate. GLAP IV supports federal government commitments and obligations under the GLWQA and COA, as well as federal government priorities outlined in the 2007 Speech from the Throne and two recent budgets. Other legislative authorities include the Canada Water Act, International Boundary Waters Treaty Act, and Canadian Environmental Protection Act, 1999.

Top of Page

3.2 Program Performance: Design and Delivery

This section presents the evaluation findings related to the adequacy of the program design and delivery from a number of aspects outlined in the evaluation matrix, including the engagement of partners, work planning activities, roles and responsibilities, resource allocations, monitoring and reporting activities, and GLSF program delivery.

Overall Findings

Overall, GLAP IV was viewed as a sensible model to achieve intended outcomes. Engagement of partners represents a critical and beneficial aspect of the GLAP IV program delivery. Partnerships occur at many levels and in many ways. Formal agreements between Canada and the United States (GLWQA) and Canada and Ontario (COA) represent the foundation for commitments of the jurisdictional parties with respect to the AOCs. Implementation involves horizontal partnerships within Environment Canada and across the federal government. RAP committees provide a forum for governments (federal, provincial, municipal and First Nations) and key stakeholder groups (conservation authorities, community groups and NGOs) to coordinate their efforts through to the delisting process. The funding structure of GLSF projects promotes partner engagement at the local and regional level with partner contributions, including cash and in-kind funding, assistance with delivery, and participation in advisory or research capacities.

The implementation of GLAP IV diverged from the original design of the program in several areas. The five-year work plans required by federal partner departments to access GLAP IV funds were considered to be a positive and worthwhile exercise. The intended work‑planning annual reporting, review and updating process did not occur as intended, however, which resulted in a loss of flexibility, responsiveness and accountability.

Due to the horizontal nature of the initiative and broad changes at Environment Canada, GLAP IV governance and management activities do not reflect those outlined in the management framework developed for the program. Areas identified as a potential source of confusion or concern by federal key informants include an inadequate instrument to ensure that partner contributions to program goals are fulfilled, and clarity of roles and responsibilities with respect to delisting AOCs.

For federal partners who received their recommended funding, the amount of GLAP IV funding was seen to be adequate, with the caveat that they would like future funding to be indexed for inflation. However, some federal partners report receiving significantly fewer resources than their original approved funding levels, which they feel negatively affected their ability to implement planned activities. For federal partners that received GLAP IV funding allocations in full, a more pressing concern was the high ratio of operations and maintenance (O&M) to salary dollars and inability to convert these dollars (i.e., from O&M to salary), which left many groups with insufficient salary dollars to hire scientific and technical staff. GLSF project proponent and committee key informants were more apt to suggest that the GLAP focus and resources be expanded to include priority areas and activities that extend beyond the designated AOCs.

GLAP IV monitoring and reporting activities are undertaken to meet annual reporting obligations associated with both the GLWQA and the COA. At the level of funded projects, GLSF project reporting is quite complete and the majority of reviewed GLSF files provided clear evidence of intended output achievements. However, limited evidence was provided on the outcomes of remedial actions taken in the AOCs. While federal proponents were supportive of increasing reporting requirements for GLAP projects to enhance performance measurement and accountability, regular reporting of federal science and monitoring projects is not a formal requirement and reporting to the GLAP IV program is ad hoc.

Project proponent key informants indicate a high level of satisfaction with the GLSF program. Program priorities are considered to be clear and appropriate. The application process is regarded as clear and transparent by most funding recipients and selection criteria are viewed to be explicit and well‑communicated. Overall, the GLSF proposal review process is perceived to be logical and GLSF funding decisions are viewed as strategic and fair. Program staff are seen by project proponents to be accessible and supportive. Modifications to the GLSF program (including the shift from the use of MOUs to contribution agreements), implemented by Environment Canada in 2007, have challenged the timely delivery of the program, reduced flexibility and increased uncertainty among partners. Project proponents indicate a preference for streamlining the application and approval process, for multi-year agreements, and for more straightforward funding agreements.

  • Evaluation Issue: Is the GLAP IV design appropriate for achieving expected program results?
  • Indicator(s):
    • Plausible link between program activities, outputs and intended outcomes
    • Clearly defined and understood governance structure, including program processes, roles, responsibilities and accountabilities
    • Program resources/capacity commensurate with expected program results
    • Views on the appropriateness of program activities, processes and governance structures
  • Methods:
    • Document review
    • Key informant interviews
    • File review
  • Rating:
    • Indicator 1 - Achieved
    • Indicator 2 - Little progress, priority for attention
    • Indicator 3 - Progress made, attention needed

Top of Page

3.2.1 Engagement of Partners

Key informants across the respondent groups had a favourable view of GLAP IV as a program model: the program was generally viewed as a sensible approach to achieve the intended outcomes. No major redundancies or notable gaps were identified within the program.

An important feature of the program, consistently highlighted by key informants across various respondent groups, is the engagement of partners and stakeholders. This occurs in many ways, perhaps most importantly in the formal agreements between Canada and the United States (GLWQA) and Canada and Ontario (COA). These agreements are the foundation for commitments of the parties with respect to the AOCs and are a critical framework in organizing work in the AOCs. Other examples of partner engagement include:

  • Multiple groups within Environment Canada, including the Canadian Wildlife Service and the NWRI. In addition, the program formally includes eight other federal government departments, which are represented on the GLEC.

  • RAP committees, which play an important role in driving work in the AOCs. The RAP three-stage reporting process is the basis for updating the GLWQA on progress in the AOCs and the delisting process, as well as for engaging local partners "on the ground." While RAP committees vary in their size and structure, key member representatives can include: federal representatives (e.g., DFO, Environment Canada), other government representatives (Ontario MOE or MNR, municipal/regional, First Nations), Conservation Authorities, community groups and NGOs.

  • GLSF projects, which require partnerships as a condition of funding approval, particularly (though not exclusively) funding partners. The review of GLSF project files confirmed that all GLSF projects include partners – six on average, with organizations having a local/regional scope of operations being engaged most often (85 per cent of projects), followed by organizations operating at the provincial level (72 per cent) (Table 3.1). Types of partners included: government (federal, provincial, municipal/regional, First Nations) (83 per cent), community-based organization (69 per cent) and the private sector (industry, landowners) and educational institutions (both at 29 per cent). The nature of partner contributions most often included cash and in-kind contributions (e.g., equipment, land, office space or supplies) (97 per cent); followed distantly by assistance with delivery (e.g., planting) or technical advice/assistance (18 per cent each), and participation in an advisory or research capacity (13 and 11 per cent, respectively). "Other" contributions include such things as publicity/promotion and training. Partnerships for federal partner projects were not analyzed in the same way, however, publication listings indicate the involvement of partners as investigators. As well, NWRI project-level documentation for the NWRI shows collaborations with municipalities, the province and other researchers. Contributions include: data sharing; sampling/provision of samples, facilities and field support; and constructing facilities. Leveraged resources are also indicated ($550,000, for two projects out of five in total).

    Table 3.1: Characteristics of GLSF Agreements: Partnerships*,**
    Scope of partners' operationsPercent of files
    * Based on a sample of 39 GLSF project files
    ** The average number of partners per project is 6
    Local/regional85%
    Provincial72%
    National51%
    International5%
    Sectors represented
    Governments83%
    Community-based69%
    Private sector29%
    Educational institutions29%
    Professional associations14%
    Other countries3%
    Other14%
    Partners' contribution
    Funding97%
    Service delivery18%
    Technical advice/assistance18%
    Advisory committee13%
    Research11%
    Not described5%
    Other13%
  • Although federal projects do not have the same requirement for partnering, 16 of 21 federal project files that were reviewed identified partners. These partners typically included Environment Canada or federal OGDs, the Province of Ontario, municipalities, a RAP committee, or university. The nature of partner contributions for federal projects focused most often on collection or sharing of samples or data and information analysis or exchange.

Summary: Engagement of partners represents a critical and beneficial aspect of GLAP IV. Partnerships occur at many levels and in many ways. Formal agreements between Canada and the United States (GLWQA) and Canada and Ontario (COA) represent the foundation for commitments of the jurisdictional parties with respect to the AOCs. Horizontal partnerships within Environment Canada and across the federal government are necessary for GLAP implementation. RAP committees provide a forum for governments (federal, provincial, municipal and First Nations) and key stakeholder groups (conservation authorities, community groups and NGOs) to coordinate their efforts through the delisting process and engaging local partners "on the ground." The funding structure of GLSF projects promotes partner engagement at the local and regional level. Partner contributions include cash and in-kind funding, assistance with delivery, and participation in an advisory or research capacity.

Top of Page

3.2.2 Work Planning

To access GLAP IV funds, federal partner departments, including Environment Canada, each submitted five-year work plans that outlined the projects they proposed to implement using GLAP IV funds and described how each aligned with program priorities. A review team (with representatives from all federal departments that submitted work plans) assessed each of the work plans. In evaluating departmental work plans, priority was given to completing federal actions in revised group 1 AOCs, while revised group 2 AOCs were accorded second priority.[30] As well, the review team did not recommend projects:

  • where a need was not clearly identified or where it was uncertain;
  • that were considered purely research;
  • that were not an essential part of a coordinated package of projects; or
  • that were not considered essential for completing federal actions for BUI restoration or AOC delisting.

The work plans were intended to steer work so that it would be responsive to the needs of the AOCs and be a tool for accountability. In this regard, recipient departments were to report on progress and update their work plans annually, at which time work plan priorities and activities could be adjusted as needed. With the endorsement of the GLEC, the Environment Canada Program Director would sign off on the allocation. While funds for the full five-year work plan period were approved in principle to provide resource stability to organizations, it was also stated that, if necessary, departments would be able to reallocate their total resources based on the reviewed and adjusted/updated work plans. Due to a Department-wide transformation beginning in 2005/06, the GLBEI (of which GLAP IV was a part) was co-led by the RDG‑Ontario and RDG‑Quebec. The intended work planning annual reporting, and review and updating process, never occurred.

Departmental managers and federal partners who were interviewed for the evaluation approved, in general, of the work planning process as a worthwhile exercise to guide activities in AOCs. However, the process as it was intended was abandoned and work plans were not reviewed annually. There was therefore no mechanism available for changing work priorities or reallocating funds on an annual basis (though this latter issue was not universally perceived to be problematic) – the approved projects became "hard‑wired" for the five-year term of the program. As well, the role of the work planning process as a tool for accountability (i.e., to ensure that activities were being conducted by the various partners according to their work plans and to address GLAP IV objectives) did not materialize.

Summary: The five-year work plans required by federal partner departments to access GLAP IV funds and proposed projects were recommended by the review team and approved by the GLEC. Federal key informants consider work planning a positive and worthwhile exercise to guide remedial and monitoring activities in the AOCs. However, the intended work‑planning annual reporting, review and updating process did not occur, which resulted in a loss of flexibility, responsiveness and accountability.

Top of Page

3.2.3 Roles and Responsibilities

GLAP IV is a horizontal and bilateral initiative, with an AOC-based RAP committee structure (described in subsection 1.1). As a result, contributions to the delivery of federal commitments in AOCs are highly distributed. Commitments of the federal and provincial governments are outlined in the COA, and the GLWQA articulates Canada‑U.S. commitments in the AOCs.

Program documentation with respect to the governance and management of GLAP IV (Management Framework: Great Lakes Action Plan for Areas of Concern (2005‑2010)) does not reflect the current operation of the program in at least one respect. As a result of the departmental transformation that occurred in 2005‑2006, the original structures and authorities for program management were not established in their entirety. The original documentation on the roles and responsibilities for the program identified a Program Management Committee that would have responsibility to support the GLEC, help facilitate the flow of information among the departments, and allow directors/managers the opportunity to brief their senior management on key issues. The Program Management Committee has not been an active committee during the past several years.[31] The communication and coordination function has fallen to the COA Annex Implementation Committee, which has a similar membership. This was not widely noted by key informants, and some respondents perceived a gap in coordination and communication across federal partners.

Despite the horizontal and bilateral nature of the initiative and broad changes in the Department, most federal key informants felt they had a good understanding of their own roles and responsibilities with respect to the program and, at a general level, those of the major partners in the program. GLSF proponents also believed they had a clear understanding of their roles and responsibilities.

However, the roles and responsibilities of federal partners was an aspect of GLAP IV that came under criticism in the CESD's reviews of the program in 2001 and 2008 and the TB MAF exercise with respect to the GLBEI in 2008. The 2008 CESD report noted that "while the department has recently clarified some responsibilities, it has still not clearly specified who is responsible for carrying out all the required remedial actions, who will pay for those efforts, and within what timelines the actions will be taken." Similarly, the Round V TB MAF exercise carried out in 2007‑08 urged the Department to clarify partner roles/contributions and to "review the change to a distributed responsibility model to deliver Great Lakes Water Quality Agreement/Canada-Ontario Agreement commitments to ensure sufficient resource allocation and the ability to meet changing priorities."

While generally aware of GLAP IV roles and responsibilities, a number of key informants echoed some of the CESD and Treasury Board concerns with the "distributed responsibility" model of this horizontal program. For example, there was a feeling among some, particularly those at the working level, that there is a disconnect (or at least the connection is not readily apparent) between the needs of the individual AOCs (as they are articulated in the RAPs) and the remedial as well as science and monitoring projects being carried out by the federal partners. Other key informants noted that, in general, communication and coordination among federal partners is an area that merits further attention from the program. It is difficult to say what type of internal management mechanism or instrument might have most benefit for the program (and key informants, when pressed, did not specify), though there is generally little enthusiasm for additional structures and meetings unless the benefits appear evident.

In response to the CESD report, the program has taken steps in 2009‑2010 to clarify the actions remaining in each AOC, the responsibility of each partner in the AOCs and the proposed timelines. This information, including an update on the status, accomplishments and the responsible partners and timeframes for remaining actions in each AOC, was presented in January 2010 to RAP committees and federal and provincial partners during one of a series of program workshops,[32] and garnered positive reviews (as heard in the evaluation interviews).

Another issue raised by some federal key informants is a grey area surrounding roles and responsibilities with respect to delisting AOCs. The GLWQA states that "The Parties shall cooperate with State and Provincial Governments to classify Areas of Concern by their stage of restoration progressing from the definition of the problems and causes, through the selection of remedial measures, to the implementation of remedial programs, the monitoring of recovery, and, when identified beneficial uses are no longer impaired and the area restored, the removal of its designation as an Area of Concern" and that "The Parties, in cooperation with State and Provincial Governments, shall ensure that the public is consulted in all actions undertaken pursuant to this Annex." In practice, the definition of delisting criteria has proven challenging because of difficulty in determining a target or end point for actions (i.e., defining "when the job is done"). The absence of clear delisting criteria for AOCs came under criticism in the 2008 CESD report as well. Several AOCs are now approaching a state of restoration, but, for some respondents, the authority and process for decision making about delisting are unclear. A small number of respondents noted the potential for a built-in disincentive to delist, due to the resulting termination of GLAP dollars for the AOC and the lack of alternative funding to monitor these areas to ensure that they remain restored. Environment Canada is currently working with the IJC to reach agreement on a consistent set of metrics for recommending restoration of BUIs and delisting of AOCs.

Summary: GLAP IV is a horizontal and bilateral initiative governed by commitments outlined in the GLWQA and COA, with an AOC-based RAP committee structure. Due to the horizontal nature of the initiative and broad changes at Environment Canada, GLAP IV governance and management activities do not reflect those outlined in the management framework developed for the GLAP for AOCs (2005‑2010). Areas identified as a potential source of confusion or concern by federal key informants include: instruments for ensuring that partner contributions occur and are coordinated and consistent with the goals of GLAP IV; and clarity of roles and responsibilities with respect to delisting AOCs.

Top of Page

3.2.4 Resources

As indicated, the original allocation for GLAP IV was $40 million over five years. The work planning process identified and recommended funding to federal partners with the intention that amounts would be reviewed and determined in a flexible manner annually, based on evolving needs and identified priorities. In the first year of GLAP IV, allocations were consistent with recommended funding levels (albeit rolled out late in the fiscal year). However, a number of federal partners within Environment Canada noted that the remaining years of GLAP IV saw a significant divergence in the allocation, with some groups such as the Canadian Wildlife Service reportedly receiving substantially less funding (though this could not be confirmed due to the absence of financial tracking information). As well, the annual review (and possible adjustment) of the funding allocation did not occur; funds for subsequent years were allocated to federal partners for the remainder of GLAP IV based on the amount of the first fiscal year allocation.

For those federal partners that received GLAP IV funds, a serious concern was the mix of salary and O&M dollars that were received. DFO and groups within Environment Canada were allocated funds that, in the first year of program implementation, had the flexibility to change O&M for salary dollars and vice versa. A Department-wide initiative subsequently froze these dollars in their respective categories. Because the original allocation was heavily weighted toward O&M, many groups with responsibility for delivering GLAP IV‑funded initiatives had the challenge of insufficient salary dollars to hire the research or technical staff required to carry out the labour-intensive science and monitoring project activities on a long-term basis.

For federal departments and groups within Environment Canada that received their original allocation, the funding amount was generally seen to be "about right," with the caveat that yearly funding amounts for future work in the AOCs would benefit from being increased to account for inflation. In contrast, GLSF project proponents were more apt to argue strongly for an increase in funding to this program component to permit a more aggressive and comprehensive approach to addressing issues in the AOCs. Funding to the GLSF was curtailed after 2005‑06.

A small number of key informants, particularly among the committee members, strongly favoured an increase in GLAP resources. However, this view also included a preference for expanding the terms and conditions of the program beyond AOCs (e.g., expanded AOC boundaries to include watersheds, heavily degraded areas around the lakes that are not AOCs, or the entire nearshore). In other words, while the current level of resourcing for GLAP IV was deemed to be sufficient for the program's narrow focus on AOCs, these respondents would like to see both the resources and programming focus expanded.

A critical issue with respect to resources and financial accountability is the very limited use of financial codes to track GLAP IV program resources. In the June 2006 departmental transformation, Environment Canada determined that priority ecosystem initiatives, including the GLBEI (of which GLAP IV is a part), would be delivered through an unbundling[33] of activities and resources. GLAP IV resources were allocated to Outcome Project Groups[34] within Environment Canada that were responsible for achieving GLAP IV objectives. However, these monies were not coded separately from Great Lakes A-Base resources, nor were they coded separately from resources dedicated to the GLBEI within the Department, thus severely limiting the financial accountability of the program.

Summary: For federal partners and groups within Environment Canada for whom resource allocations were consistent with their original recommended funding levels, allocated GLAP IV funding was viewed to be adequate, with the caveat that future funding should be indexed for inflation. However, some federal groups received significantly less resources than their original approved funding levels, thus negatively affecting their ability to implement planned activities. For federal partners and groups within Environment Canada that received GLAP IV funds, a more pressing concern than the level of funding was the high ratio of O&M to salary dollars, which left many groups with insufficient salary dollars to hire the staff required to undertake science and monitoring project activities. With direct transfer of funds to partners, but no program‑specific coding of expenditures, financial accountability is a significant weakness of the program. GLSF project proponent and committee key informants were more apt to suggest that the GLAP focus and resources be expanded to include priority areas and activities that extend beyond the designated AOCs.

Top of Page

3.2.5 Monitoring and Reporting

There are reporting obligations associated with the GLWQA and COA to which GLAP IV must respond. The GLWQA requires the parties (Canada and the United States) to report progress on AOCs to the IJC. Comprehensive reporting on AOCs was provided in 2003 to the IJC (resulting in the April 2003 IJC report Status of Restoration Activities in Great Lakes Areas of Concern: A Special Report). An informal briefing on the status of AOCs was also provided to the Canadian Section of the IJC (the Canadian secretary and IJC staff members) in 2009. The Report urged Canada and the United States to: fulfill their commitments to report on "...progress in developing and implementing Remedial Action Plans and in restoring beneficial uses..."; "...document their considerable investment and achievements to date in order to provide the public with a true reflection of their accomplishments."; and "...ensure that monitoring, data support and information management systems are in place..."

Reporting of GLAP IV also occurs annually to update delivery of federal commitments in the COA. Progress in relation to achievement of COA commitments is assessed annually and reported to the COA Management Committee, based on an extensive COA database, where federal and provincial parties input all their projects against each COA result number as per the COA agreement.

At the level of funded projects, GLSF projects follow a reporting template that includes progress reporting (where required) and a final report (or annual report for multi-year projects). Final reports include background/project description, a listing of activities/outputs against intended deliverables, key products (e.g., published papers, research documents), highlights of the project, as well as final financial reporting on project expenditures (GLSF funding and leveraged contributions). The file review confirmed that GLSF project reporting is quite complete: the majority of files contained documentation relevant to the purpose of the project (proposal, technical review), MOU or contribution agreement, as well as a final report and associated products. The file review further confirmed that in the vast majority of files (80 per cent), evidence of intended output achievement was provided to a very or moderately clear degree in the documentation. Three quarters of projects were rated to have a very close or moderate alignment between the project's design and implementation.

Several key informants in various respondent groups pointed out that a challenge in reporting on GLSF-funded projects is measuring the outcomes of the remedial actions taken in the AOCs. While habitat or remediation initiatives may be considered a success in terms of outputs, it is much more difficult to determine the level of success in terms of outcomes (e.g., BUI and AOC restoration). Indeed, the review of files indicated that projects were more likely to have output-related objectives than outcome-related objectives: 38 per cent of files contained output-related objectives only; 15 per cent contained outcome-related objectives only; and 46 per cent contained both types of statements.[35]

As well, the files were far more likely to provide evidence of achievement of intended outputs (80 per cent clear or moderately clear evidence of outputs) than outcomes (54 per cent clear or moderately clear evidence of outcomes). Material on outcomes was more frequently available for projects that had a monitoring component (e.g., water quality or population data) or that were conducted as part of an ongoing multi-year effort. External evaluations of project results were rare.

Reporting requirements for federal science and monitoring projects under GLAP IV are not specified to the same extent as GLSF projects. As a result and as noted previously, the review of federal projects was addressed in a qualitative and descriptive fashion. Reporting on federal projects may include reporting / annual reports to the RAP committees (for whom the science or monitoring was conducted), GLWQA/COA reporting, and contributions to the scientific literature (papers and presentations). For projects conducted by DFO, for example, almost 70 conference presentations were prepared and over 25 reports and publications were prepared for the sample of nine projects funded under GLAP IV.

Regular and structured reporting to the program on project activities has not been made a formal requirement of the funding, and therefore reporting has been sporadic. Federal partners noted that periodic workshop presentations or summaries of activities were requested and were provided to the program on an ad hoc basis.

Federal proponents were generally supportive of increasing reporting requirements for GLAP IV projects, a trend they viewed as being consistent with overall increased attention in government to performance measurement and accountability. While there are some projects, such as basic science, for which performance measures may be difficult to develop for outcomes, respondents were, for the most part, agreeable to enhanced reporting on deliverables outlined in their work plan.

An aspect of reporting that was raised by a small number of key informants was the absence of a mechanism to share the results of science and monitoring activities carried out in the Great Lakes AOCs in order to facilitate technical transfer and share lessons learned. This was a commitment that was set out in the COA Annex 4 pertaining to the AOCs, but was never fulfilled.[36] The development of web-based information-sharing would fall under the Internet Content Renovation Initiative (formerly the One Department, One Website initiative), which moves the Department to a more centralized approval process for publishing of web content. With this, the Ecosystem Sustainability Board is responsible for approving all new web content, and the COA commitment has not been prioritized and approved by the Board. However, GLAP IV has held workshops to facilitate technical transfer. Finally, the lack of inclusion of members of the public or lay audiences in plans for dissemination of activities and results of GLAP IV work in the AOCs was also noted by a small number of key informants as a deficit in program communications.

Summary: GLAP IV monitoring and reporting activities are undertaken to meet annual reporting obligations associated with the GLWQA and COA. At the level of funded projects, GLSF project reporting is quite complete. The vast majority of reviewed GLSF files provided clear evidence of intended output achievements, however, only limited evidence was provided on the outcomes of remedial actions taken in the AOCs. Regular reporting of federal science and monitoring projects is not a formal requirement of the funding program, and reporting to the GLAP IV program over the period reviewed has been ad hoc. Federal proponents were generally supportive of increasing reporting requirements for GLAP projects in order to enhance performance measurement and accountability. There was a weakness identified in information sharing among GLAP IV stakeholders and the general public.

Top of Page

3.2.6 GLSF Program Delivery

Feedback from GLSF funding recipients indicated a high level of satisfaction with the design and delivery of the program. Almost all project proponent interviewees considered GLSF priorities to be clear and appropriate. GLSF projects are perceived to be aligned with activities outlined in the RAPs developed for the respective AOCs, and are seen by interviewed project proponents as contributing to the overall health of the Great Lakes.

Nearly all project proponents considered the GLSF application process to be clear and transparent. Selection criteria were viewed by most funding recipients to be explicit and well-communicated, with direct links to GLSF priority areas and to delisting criteria indicated in their AOC's RAPs. The GLSF proposal review process was perceived to be logical and GLSF funding decisions are considered to be strategic and fair. According to one project proponent, GLSF is "one of the better-run programs – very focused."

For the majority of funding recipients, the proposal review process was seen as a collaborative and consultative exercise that provides for two-way conversations on the alignment of proposed activities with GLAP IV and RAP priorities and needs. While almost all interviewees were satisfied with the clarity and transparency of the application process, a few project proponents found the process to be overly complex and, depending on the scope and technical nature of the project, noted that it could represent a disincentive for organizations without appropriate in-house technical support, or for smaller projects where the value of the funding sought may not justify the level of effort expended to apply for funding.

The support provided by program staff during the proposal development and review process was seen by nearly all GLSF proponent interviewees as being constructive, with a good level of interaction and support provided. As one project proponent indicated, it is "a very interactive and beneficial process." Most interviewed GLSF recipients indicated that program staff are accessible during the proposal development process, and the feedback and support provided by project officers is helpful. For some project proponents, Environment Canada support enabled them to improve their proposal by enhancing the alignment of the project with the goals and objectives of GLAP IV or by providing technical support for high-profile projects in binational AOCs. A few project proponent interviewees experienced a decline in the level of support provided by program staff, which was attributed to high turnover rates of GLSF staff that occurred during a 2007‑08 program‑restructuring initiative. However, these interviewees also noted recent improvements in the level of support provided by program staff.

While there is a high level of satisfaction in the overall design and delivery of the GLSF program, there is also a high level of consistency among project proponents in the perceived weaknesses of the current design and delivery of the GLSF program. A number of changes to the GLSF approval and funding processes were implemented by Environment Canada in 2007. Notably, the MOU mechanism traditionally used to fund GLSF projects using O&M resources was replaced by contribution agreements to support projects using G&C funds, and the approval process was amended to require senior management and ministerial authorization. The contribution agreement funding mechanism (and associated approval requirements) has compromised the timely start‑up of some time-sensitive on-the-ground projects. Other concerns noted by project proponents included unexpected decreases in GLSF funding, which has led some to consider cancelling or limiting the scope of their projects, and some lack of clarity in the GLSF agreement's numerous clauses.

A majority of funding recipients believe Environment Canada should implement a more streamlined application and approval process to enhance the efficient delivery of the GLSF program by reducing the level of effort expended to access funding (e.g., simplify the application form for non-technical or low-dollar‑value projects, and enable "umbrella" annual proposal submissions by organization) and by shortening the amount of time needed to approve funding proposals and agreements (i.e., simplify the internal review and approval process). A minority of funded recipients indicate a preference for multi‑year agreements that would increase the effectiveness of on-the-ground projects by enabling multi-year project planning, thus ensuring timely start-up of field work by avoiding yearly approval processes.

The issues raised by GLSF project proponents about G&C approval processes are consistent with those identified by the Blue Ribbon Panel on Grants and Contributions in 2006.[37] Since that time, federal departments (including Environment Canada) have been undertaking grants and contributions reform initiatives, which include action plans to, for example, become more client-focused, such as through making the funding process and programs more transparent, reducing the administrative burden on funding recipients, and providing funding to clients on a more timely basis.[38]

Summary: Project proponent key informants indicate a high level of satisfaction with the GLSF program. Program priorities are considered to be clear and appropriate. The application process is regarded as clear and transparent by most funding recipients and selection criteria are viewed to be explicit and well‑communicated. Overall, the GLSF proposal review process is perceived to be logical and GLSF funding decisions are viewed as strategic and fair. Program staff are seen by project proponents to be accessible and supportive. Modifications to the GLSF program, implemented in 2007 by Environment Canada, have challenged the timely delivery of the program, reduced flexibility and increased uncertainty among partners. Project proponents indicate a preference for streamlining the application and approval process, for multi-year agreements, and for more straightforward funding agreements.

Top of Page

3.3 Program Performance: Achievement of Program Outcomes

This section presents the evaluation findings related to the achievement of the program's intended outcomes, and the identification of any unintended impacts of GLAP IV. Information is also provided on the overall importance of the contributions made by GLAP IV to the restoration and maintenance of AOCs, as well as the identification of external factors that may be affecting, either positively or negatively, the program's activities and outcomes.

Overall Findings

In general, key informants had favourable impressions about the performance of the program with respect to the achievement of immediate and intermediate outcomes, a finding that is supported by the program documentation and review of GLSF and federal project files. As indicated, engagement of partners at the local level and across jurisdictions and scientific communities is perceived to be a strength of the program. This is supported by GLAP IV funds allocated to federal partners to facilitate coordination and management of GLAP IV (e.g., with the province and First Nations). Engagement of partners and participation at the local level is facilitated through committees, informal networks and the RAP structure (which also receive GLSF funding support). With respect to remedial actions, addressing pollution (through sediment remediation, support to municipal infrastructure improvements, and reducing non-point sources of pollution) is a significant priority for the program and is allocated almost one half of the GLAP IV dollars. While the success of federal partners' efforts is difficult to discern due to gaps in the federal project files, GLSF projects that supported this objective were found to be well‑documented in terms of outputs and experienced few challenges in implementation. BUI assessment and monitoring work is occurring on many fronts to assess the status of BUIs and the effectiveness of restoration activities, and to define BUI goals/targets. Fewer federal projects were funded for habitat restoration (though this is a particular focus of GLSF). Habitat projects are more easily documented in terms of outputs (e.g., plantings, wetland acreage), but these projects are also more subject to implementation challenges. Overall, program investments advance activity in each AOC to some degree (though some coordination projects target all AOCs generically). Note that while group 1 AOCs (those closest to delisting) were intended to be assigned a higher priority for GLAP IV investments, the number of projects funded in these group 1 AOCs is lower than in the group 2 AOCs.

Achievement of the program's longer-term outcomes is much less evident and the majority of BUIs that were originally identified in AOCs still exist. Of the over 100 BUIs identified, only 20 per cent have been restored. In consideration of the status of the BUIs, the program has identified outstanding priority actions for each AOC and projected timelines for delisting. Only two AOCs have been delisted and one AOC designated an area in recovery, and these occurred prior to GLAP IV. The original program goal of completing federal actions in seven group 1 AOCs has not been achieved. Three AOCs are expected to be delisted or designated as Areas in Recovery within the next 1‑2 years, and a fourth, Wheatley Harbour, has just been delisted (announced on April 16, 2010). With respect to the remaining AOCs, the picture is much more complex and the time frames for delisting are longer-term, with most of these AOCs estimating delisting as occurring between 2015 and 2020. For committee and expert key informants, the overall importance of the contribution made by GLAP IV to the restoration and maintenance of AOCs is viewed as being very significant, if not critical.

For key informants, positive external factors that support program success include: leveraged funding, particularly infrastructure funding; U.S. investments; and community/political engagement (for example, the Great Lakes St. Lawrence Cities Initiative). External factors that interviewees identified as having the potential to negatively affect the success of GLAP IV include: new and changing ecosystem issues, and the economic downturn. Unintended outcomes of GLAP IV cited by key informants tend to be positive, and focus on unexpected interest and engagement of non-targeted groups (e.g., landowners) and the general public as well as on unanticipated opportunities for collaboration and knowledge transfer.

Top of Page

3.3.1 Program Outcomes

  • Evaluation Issue: To what extent have intended outcomes been achieved as a result of the GLAP IV?
  • Indicator(s):
    • Evidence of/views on intended output and outcome achievement
    • Evidence of/views on factors outside the program that have influenced the achievement of intended outcomes
    • Views on the extent to which intended outcomes have been achieved as a result of the program
  • Methods:
    • Document review
    • Key informant interviews
    • File review
  • Rating:
    • Progress made, attention needed
Immediate and Intermediate Outcomes

The extent to which GLAP IV has contributed to intended outcomes was explored in key informant interviews, as well as the review of program documents and the file review.[39] In general, key informants who were asked about program-level outcomes had favourable impressions of the performance of the program regarding shorter-term outcomes. Building on the achievements of previous GLAP programs, much work has already been accomplished in identification and characterization of problems in AOCs. The assessment of progress toward achievement of intended shorter-term outcomes identified in the program logic model is based on the perceptions of key informant interviewees, as well as the program documents and files. While descriptions of and allocations to federal projects have been reviewed (i.e., from the original approved work plans), there is limited evidence on the outputs and success of these efforts. Evidence related to the achievement of each of the program's immediate and intermediate outcomes is as follows:

  • More effective and better‑integrated remedial actions in AOCs. GLAP IV has supported effective and integrated remedial actions in AOCs in a number of ways:
    • RAP committees, as the basis for links between the federal level and community/provincial stakeholders, ensure that environmental needs at the AOC level are addressed. The committees work within the RAP reporting framework as outlined in the GLWQA. While the RAP committees predate GLAP IV, the program continues to support the committees in AOCs where there is interest and capacity. According to key informants, the structure of RAP committees varies across the AOCs, as does their level of activity and engagement. In some AOCs (e.g., northern Lake Superior locations), some RAP committees were inactive for quite some time. In other locations such as the St. Lawrence and Bay of Quinte, the RAP committees are very active and integrally involved in remedial actions.
    • GLSF program officers and other federal partners (depending on the type of expertise required) participate on RAP committees and play a leadership role in those AOCs where the federal level is identified as the lead.
    • Each year, a portion of GLSF funds is used to support RAP committees. For example, in 2008‑09, $180,500 in contribution dollars (nine per cent of GLSF funds in that fiscal year) was allocated to AOC RAP‑related coordination and governance in four AOCs. In 2007‑08, $470,500 (18 per cent of GLSF funds in that fiscal year) was allocated to governance activities in six AOCs.
    • Nearly one in ten GLSF‑funded projects reviewed focused on management/coordination, and of these, two thirds provided evidence of output and outcomes achievement.
    • Among federal projects, ECB was allocated $318,000 annually for RAP coordination. This activity included "collaborative action among government, organizations and basin residents …[leading to] functioning implementation frameworks."
    • In addition to RAP committees and as noted by key informants, effective and integrated remedial actions are facilitated through the work of formal committees (e.g., the COA Annex Implementation Committee) and informal networks of technical and scientific professionals working in the Great Lakes.
  • Improved identification of environmental problems and progress in AOCs. Identification of environmental problems in the AOCs has largely been accomplished in previous GLAP iterations, through the development of the Stage 1 RAP reports. However, additional work has been carried out in GLAP IV to update the status of environmental problems in selected AOCs and further characterize their nature through research (e.g., characterization of sediment in Randle Reef). A noted benefit of GLAP IV funds is the ability to expand problem identification and monitoring beyond narrow municipal boundaries in order to undertake more comprehensive, system-wide assessments. Ongoing monitoring of the status of BUIs is an important component of work under GLAP IV:
    • This is an area where federal partners contribute greatly. For example, among the projects recommended for funding, approximately $11 million was allocated to BUI assessment and monitoring.
    • In the sample of GLSF files that was reviewed, almost six in ten files had objectives related to assessment and monitoring of water quality (26 per cent), contaminated sediment (23 per cent), or ecosystem health based on BUIs (10 per cent). Outputs are moderately well‑documented in the files.
  • Improved management and coordination of efforts to restore and maintain the Great Lakes Basin ecosystem. As indicated, engagement of partners was identified as a strength of the GLAP model. A committee structure predates GLAP IV, but has continued under this iteration of the program, including binational (GLWQA and the Great Lakes Binational Executive Committee, coordination in binational AOCs) and federal/provincial coordination through the COA and associated committees. Other federal‑partner efforts in the area of management and coordination included the following:
    • ECB was funded for management and coordination of GLAP IV, which involved activities and reporting to ensure that federal actions are implemented in coordination with initiatives of other organizations and to facilitate collaboration with stakeholders (e.g., $207,000 was allocated annually to activity coordination and annual reporting on the implementation of COA annexes, $3,000 annually to collaborate with First Nations communities, and $57,000 annually for meetings between federal and provincial agencies on water initiatives).
    • Federal efforts (led by ECB) were dedicated to the development of tools and methods that could be used across AOCs to facilitate consistent monitoring. Just under $1 million was allocated over GLAP IV's five-year term to "coordinated and efficient federal/provincial scientific monitoring, including leading collaborative efforts and technology transfer." This included development and implementation of monitoring plans using consistent delisting criteria, collection of inter‑agency – compatible environmental quality information, and adoption of common protocols among agencies for water quality assessments.
  • Pollution from identified sources is minimized or eliminated in AOCs. Sediment remediation is an important focus of GLAP IV, with $13.7 million initially allocated to this priority. This includes federal projects dedicated to sediment remediation (e.g., contaminated sediment risk‑management assessment, and studies and plans for harbour sediment remediation). A small number of GLSF projects in the file review addressed non-point sources of pollution. With respect to industrial pollution and wastewater management, GLAP IV plays a supportive, though important, role:
    • Almost $6 million over the five-year GLAP IV term was dedicated to "support to municipalities to implement RAP infrastructure recommendations for sewage treatment, combined sewer overflows and stormwater management." The nature of support includes scientific and technical studies to position municipalities for infrastructure funding.
    • In the review of the sample of GLSF files, municipal infrastructure support projects represented 26 per cent of reviewed files. The GLSF funds projects to support municipal applications for infrastructure improvements to wastewater treatment plants, sediment identification, and evaluation of innovative, cost‑effective technologies. Outputs for these projects were found to be well-identified in the file review, with few challenges in implementation.
  • Habitats in AOC ecosystems are restored. Just over $7 million was dedicated in the original work plan to "… rehabilitation of fish and wildlife habitat through the development and implementation of rehabilitation strategies and fish management plans. Activities include wetland creation/enhancement, coastal and stream rehabilitation, fish barrier removal, colonial water birds, stewardship, riparian and upland plantings, project management and monitoring, and protection through incorporation of strategies into municipal operating procedures." (From the GLAP IV Workplan.)
    • A large portion of these funds are allocated through the GLSF. Approximately half of the projects included in the review of the sample of GLSF files had objectives related to habitat restoration (e.g., information for municipalities on how best to protect and restore their habitats, and on shrub and tree planting, wetland creation, and shoreline stabilization). The vast majority of these projects (over 80 per cent) were able to demonstrate project outputs in the file documentation, though habitat projects were also more apt to suffer challenges in implementation (e.g., due to weather-related issues or the voluntary nature of landowner commitments).
    • Funds were also made available to federal projects to support the development and demonstration of new methods and technology transfer across AOCs and regions, and to support collaboration with local implementation agencies and OGDs.
  • Activities of federal partners and stakeholders advance remedial actions, monitoring, outreach and engagement in each AOC. Activities are occuring on many fronts, and the program tries to ensure some progress is made in all AOCs. This is confirmed by the GLSF file review: in only one AOC, Jackfish Bay, was no project funded. Group 1 AOCs accounted for 21 per cent of projects that were reviewed using the COA definition and 43 per cent of reviewed projects when using the work plan review team definition. Similarly, for federal projects, many cut across several or even all AOCs and, as a result, federal efforts support all AOCs to some extent. In addition, GLSF projects have engaged numerous partners, with each project having an average of six partners (including First Nations, community and academic groups, and the private sector, among others). Federal partners have also engaged other stakeholders such as provincial and U.S. counterparts and other government departments, to steer their efforts toward program priorities.

Top of Page

Long-Term Outcomes

The long‑term outcomes for the GLAP IV are that:

  • beneficial uses are determined to be unimpaired and AOCs are delisted; and
  • Canada's international commitments related to Great Lakes AOCs are met.
Restoration of BUIs and Delisting of AOCs

Achievement of the program's longer-term outcomes was an area where the CESD (2008) was critical of the program: "After more than 20 years, only 2 of Canada's original 17 areas of concern have been delisted [Collingwood Harbour and Severn Sound] – the latest in 2003. Priority actions have been completed for one other area, which is now recognized by the government as an ‘area in recovery [Spanish Harbour].' The majority of impairments to beneficial uses that were originally identified in areas of concern still exist today."

Current program data indicate that while many BUIs (21) are assessed as being restored, many more remain impaired (over 80 BUIs across the 15 AOCs) (see Annex I, under separate cover).[40] However, a simple binary restored/impaired measure is not a highly sensitive measure of progress toward restoration and, in some cases, BUIs in various areas may be in the process of being monitored to assess impairment (i.e., to update information on extent/nature of impairment).

The status of BUIs is indicative of overall progress toward delisting of AOCs. Recall that the goal of GLAP IV was to complete federal actions in group 1 AOCs (closest to delisting). Of these seven originally identified AOCs, only three are expected to be delisted or designated an area in recovery by the end of GLAP IV.

Using the work plan Review Committee group 1 priority grouping, three of the seven group 1 priority AOCs were expected to be delisted or designated areas in recovery within the next 1‑2 years: Wheatley Harbour has just been delisted (announced on April 16, 2010); Nipigon Bay is scheduled for delisting in 2011 or 2012, pending infrastructure upgrades; and the St. Lawrence River (Cornwall) AOC has reached the Stage 3 RAP reporting phase and is also scheduled for delisting in 2010 (see Annex I, under separate cover).

With respect to priority group 2 AOCs, the picture is more complex and the timeframes for delisting are longer-term. Several AOCs have now prepared Stage 2 RAP reports or updates (e.g., Detroit River, Niagara River). There are a cluster of AOCs such as Hamilton Harbour and Toronto and Region that require large infrastructure projects to address BUIs in the area. Finally, areas such as St. Mary's River continue with assessment, monitoring and community engagement (Annex I, under separate cover).

While restoration of BUIs and delisting of AOCs has been slower than anticipated, key informants consistently noted the significant challenges of ecological restoration work: heavily degraded areas such as city harbours require expensive infrastructure solutions; ecosystems such as the Great Lakes are complex and dynamic, requiring significant time to diagnose problems, identify solutions and test their effectiveness; and as scientific knowledge evolves, new problems or the need for new solutions are often revealed.

Key informants also highlight the notable successes in several of the AOCs (e.g., Wheatley Harbour and the St. Lawrence River), and the overall importance of the contribution made by GLAP IV to the restoration and maintenance of AOCs. For example, experts and committee member respondents note that GLAP IV funds "focus attention on AOCs," and they generally underscore that federal involvement through GLAP IV funding and scientific expertise is a critical catalyst, stating that "there wouldn't be any movement without it", or that "any holdback now would kill momentum" and that "there would have been [continued] degradation if not for GLAP."

Respondents also noted that despite the critical role played by GLAP IV in the restoration of AOCs, it is not an exclusive one. In monetary terms, GLAP IV funds represent a relatively modest contribution in comparison to larger-scale infrastructure investments. In 2003, the Government of Canada estimated that wastewater and infrastructure improvements across the AOCs would cost approximately $2 billion.

Meeting Canada's International Commitments Related to Great Lakes AOCs

The IJC assists with implementation of the GLWQA, which sets out a series of commitments for Canada and the United States – including a requirement that both countries take remedial action in the heavily degraded AOCs. Using the RAP reporting process, Canada and the United States, as parties to the Agreement, are directed to classify AOCs by their stage of restoration, progressing from

  • the definition of the problems and causes (Stage 1 RAP),
  • the selection and implementation of remedial measures (Stage 2 RAP),
  • the monitoring of recovery, and when, identified beneficial uses are no longer impaired and the area is restored, the removal of its designation as an Area of Concern (Stage 3 RAP).

RAPs were developed for all AOCs in the late 1980s when the GLWQA was revised. RAPs are submitted to the IJC for review and comment at three stages. The IJC also adopted an initiative involving status assessments to examine progress toward restoration of beneficial uses in individual AOCs, in an effort to enhance the restoration process. To date, five AOCs remain at the Stage 1 RAP level, though status assessments have been completed for the Detroit River and St. Mary's River AOCs. Stage 2 RAPs have been submitted to the IJC for eight Canadian AOCs and a Stage 3 RAP has been submitted for review for one AOC, Wheatley Harbour (Table 3.2).

Table 3.2: Summary of Remedial Action Plan Status
Stage 1Stage 2Stage 3
Source: www.ijc.org/rel/boards/annex2/rap_process.htm
Jackfish Bay
Detroit River
St. Mary's River
Peninsula Harbour
Thunder Bay
Bay of Quinte
Toronto and Region
St. Clair River
Spanish Harbour
St. Lawrence River (Ontario)
Niagara River (Ontario)
Hamilton Harbour
Nipigon Bay
Wheatley Harbour
(now delisted, April 2010)

Top of Page

External Factors

Interview respondents across all categories were asked to identify any external factors that might have a positive or negative impact on the restoration and/or maintenance of AOCs. Positive external factors that were seen to have supported program successes include:

  • External funding sources: Several interview respondents note that the availability of infrastructure funding (e.g., through Infrastructure Canada and municipalities) is critical to improvements in many AOCs, addressing costly upgrades such as improvements to sewage treatment plants (e.g., Nipigon Bay, Cornwall). Conversely, in locations where municipalities are unable to secure sufficient funding or cannot meet the matching funding requirements of infrastructure programs (e.g., Hamilton), capital projects cannot proceed. Also, leveraging of funds from external partners through GLSF projects was noted as a positive factor that enhances the scope of activities accomplished in AOCs. Partner relationships can also leverage technical expertise as well as public focus and engagement.
  • Community/political engagement: A number of interview respondents noted that community support for restoration efforts is a factor that has the potential to affect success. As an example, restoration efforts in Hamilton Harbour have led to a rejuvenation and renewed use of the waterfront by residents, which has in turn increased public interest in and support for restoration efforts. Another example of the positive impact of community engagement identified by several committee members is the Great Lakes‑St. Lawrence Cities Initiative. This initiative, which involves Great Lakes and Seaway city representatives, was identified as a positive external factor that has resulted in discussions between this group and the provincial government on water conditions and issues of particular interest to the group (e.g., algae growth, beach foulings, sewer discharge and naturalization of the shoreline).
  • U.S. investments: A number of interview respondents note that recent funding announcements by the U.S. government for restoration and maintenance in the Great Lakes may influence Canada to similarly continue and/or increase its own investments.[41]

External factors that may have had a negative impact on the success of GLAP IV include:

  • New challenges: Several interview respondents note that the Great Lakes are a dynamic ecosystem that is constantly changing, with new issues cropping up unexpectedly – such as new invasive species (e.g., Goby fish, zebra mussels) and the impact of climate change. These changes may alter targets or introduce new challenges, requiring flexibility in the approach to restoration and/or maintenance.
  • Lack of public engagement: While public support is identified as a positive factor influencing success, limited communications and weak public support or engagement is similarly identified as a negative factor by several interview respondents. These interview respondents identify a need for greater communications to the public on the remediation that needs to be done, efforts currently under way, and value/impacts of this work. These respondents believe that increased public support may also translate into enhanced political will and action.
  • Economic downturn: The economic downturn was cited by a few interview respondents as having a negative impact on GLAP IV efforts, by making it more difficult to obtain partners and financial support for projects.

Other challenges or negative external factors identified by respondents include the challenge of having a number of policy renewal activities (i.e., for the GLWQA and COA), which diverts attention from implementation work; and the challenge of restoring and delisting binational sites, due to the jurisdictional complexity and to the generally slower pace of work to date on the U.S. side of the Detroit River.

The GLSF file review also revealed that, on a small number of projects (six), challenges to implementation were identified. These challenges primarily involved project delays, due to issues such as staff turnover, site conditions, and reliance on volunteer effort.

Summary

In general, key informants had favourable impressions on the performance of the program with respect to shorter-term outcomes, and this is confirmed by the program documentation and files. Engagement of partners and participation at the local level, a perceived strength of the program, is advanced through various projects to support overall management and coordination, as well as the RAP committee structure. Addressing pollution (through sediment remediation and municipal infrastructure support) is a high priority for the program, and there have been some achievements in this area. BUI assessment and monitoring are ongoing activities that provide scientific support. Habitat restoration is a somewhat lower priority. Priority funding for group 1 AOCs is not currently reflected in the number of projects funded in these AOCs.

Achievement of the program's longer-term outcomes is much less evident, as the majority of impairments to beneficial uses that were originally identified in AOCs still exist. In consideration of the status of the BUIs, the program has identified outstanding priority actions for each AOC and projected timelines for delisting. Three AOCs are expected to be delisted or designated as Areas in Recovery within the next 1‑2 years, and a fourth, Wheatley Harbour, has just been delisted (announced on April 16, 2010). With respect to the remaining AOCs, the picture is much more complex and the time frames for delisting are longer-term, with most of these AOCs estimating delisting as occurring between 2015 and 2020. For committee and expert key informants, the overall importance of the contribution made by GLAP IV to the restoration and maintenance of AOCs is viewed as being very significant, if not critical.

For key informants, positive external factors that support program success include: leveraged funding; U.S. investments; and community/political engagement. Identified external factors that may negatively affect the success of GLAP IV include: new and changing ecosystem issues; and the economic downturn.

Top of Page

Unintended Impacts
  • Evaluation Issue: Have there been any unintended (positive or negative) outcomes?
  • Indicator(s):
    • Presence/absence of unintended outcomes
    • Opinions of key informants on whether unintended outcomes occurred
  • Methods:
    • Document review
    • Key-informant Interviews
  • Rating: N/A

Few unintended impacts were identified, either in key‑informant interviews or through the GLSF file review. Five percent of the projects reviewed in the file review identified unintended positive impacts (e.g., school-based partnerships that yielded interest in field courses and the potential to adopt material into the curriculum, as well as opportunities for additional water sampling due to above-average precipitation).

Most unintended impacts identified by interview respondents are positive and involve unexpected public interest in AOC projects or interest among the non-targeted community or groups (e.g., interest and engagement of non-participating landowners in habitat restoration initiatives), and unanticipated opportunities for collaboration among partners or stakeholders.

A second unintended impact noted by a small number of key informants was technical or knowledge transfer to other aquatic ecosystem or other research areas (e.g., tall‑grass seed production and sales arising from habitat work, as well as site habitation data used to identify potential sites for fish nurseries).

Summary: Unintended outcomes of GLAP IV identified by key informants tend to focus on unexpected interest and engagement of non-targeted groups (e.g., landowners) and the general public, and on unanticipated opportunities for collaboration and knowledge transfer.

Top of Page

3.4 Program Performance: Efficiency

This section examines the efficiency of GLAP IV activities and delivery, i.e., whether the program is undertaking activities and delivering products in an efficient manner. These evaluation findings include observations on resources leveraged through GLAP IV and administrative costs, as well as perceptions of the efficiency of project implementation and program delivery mechanisms.

Overall Findings

The cost-efficiency of GLAP IV is difficult to determine with existing information, due to the unbundling of GLBEI funds and activities and the introduction of a shared, results‑based accountability approach to environmental initiatives. The impact on financial data availability resulting from changes implemented to Environment Canada's resource allocation and accountability structure is that GLAP IV funds are not specifically coded at the program level and the A-base support allocated to GLAP IV is not known.

Financial analysis of the GLSF program component indicates that for every contribution dollar, $0.24 is spent on program administration (salary and O&M). This amount compares favourably to the EcoAction Community Funding Program, which has a cost‑efficiency ratio of $0.39, but unfavourably with the Habitat Stewardship Program ($0.08) or the Invasive Alien Species Partnership Program ($0.13). Like EcoAction, the delivery of the GLSF involves additional staff responsibilities such as participation in RAP committees.

For key informants, there is an impression of efficient program delivery both overall and at the project level, with a number of factors identified that support cost‑efficiency at the program and project levels. Some suggestions to improve efficiency were nonetheless received.

  • Evaluation Issue: Is GLAP IV undertaking activities and delivering products in the most efficient manner?
    • How could the efficiency of the program's activities be improved?
    • Are there alternative, more efficient ways of delivering the program?
  • Indicator(s):
    • Comparison of program activities and products delivered by other similar programs
    • Resources leveraged from GLAP contributions and their perceived impact on the program/funded projects
    • Analysis of actual program operational costs in relation to the production of outputs
    • Views on whether the cost of producing program outputs is as low as possible
    • Views on how the efficiency of program activities could be improved
    • Views on whether there are alternative, more efficient ways of delivering program activities and outputs
  • Methods:
    • Document review
    • Key informant interviews
    • File review
    • Financial analysis
  • Rating: Progress made, attention needed

This section presents evidence related to two indicators of cost-efficiency examined in this report – leveraging and administrative efficiency – and discusses challenges associated with each of these analyses.

Leveraging

Leveraging refers to the total value of non-federal contributions to GLAP IV projects. A key challenge related to the calculation of leveraging in the context of GLAP IV concerns the fact that total project costs and partner contributions have not been updated in the electronic program database files since 2006‑2007.

The file review, using a sample of GLSF files, does provide some information on leveraged funding for GLSF projects (Table 3.3). There is a wide range in the value of contributions – from a low of $6,500 to a high of $267,000. Consistent with program guidelines, Environment Canada contribution funding represents approximately one third of the total program cost. A portion of files, 29 per cent, was funded at a higher level, but about half of these are within a few percentage points of the one-third funding ratio and most of the remainder are projects funding RAP implementation.

Table 3.3: Environment Canada Funding and Leveraged Funding*
Amount of Environment Canada Contribution Agreement FundingPercent of Files
* Based on a sample of 39 GLSF project files
< $50,00036%
$50,000‑$100,00038%
> $100,00026%
Average contribution$76,678
Total Amount of Funding/Total Cost of Project
< $100,00028%
$100,000‑$250,00031%
>$250,00042%
Average Total Cost of Project$225,940
Ratio of Funding to Cost
< 33%56%
33%15%
> 33%29%
Average32%

Project proponents also noted the importance of GLSF resources as a catalyst for leveraged funding. Consistent with the importance of collaboration in Great Lakes restoration and maintenance issues, one key informant confirms that the "federal level has been particularly effective in providing seed money, for example, GLSF provides 1/3 [funding], as well as a stamp of approval, enabling leveraging from other sources." RAP committee key informants identified other ways in which GLAP IV funding served as a catalyst by noting that "GLAP money has been essential to support the Remedial Action Planning which involves many stakeholders" and federal involvement is "important in terms of guidance, experience, coordination, and focusing the efforts."

Unfunded GLSF applicants were asked to indicate whether their project proceeded despite the lack of success in obtaining funding. One of three unfunded applicants interviewed indicates that their project went ahead as planned with several smaller sources of funding. Two of three unfunded applicants interviewed note that their lack of success in obtaining funding has set their projects behind; the projects continue but have been delayed and are proceeding more slowly as they try to secure other funding sources.

Federal projects were far less likely to have leveraged resources from partners. Of the 21 sampled federal project files, only two indicated that the project involved leveraged resources from partners. Leveraged resources therefore represent a small fraction of total project costs (approximately five percent).

Administrative Efficiency

Another indicator of program efficiency is derived by examining administrative efficiency (the ratio of operational costs to program dollars). Again, there are limitations in the extent to which this analysis can be conducted, due to departmental changes that occurred during GLAP IV. As indicated, beginning in 2005‑2006 the Department transitioned to a shared, results-based accountability approach to ecosystems initiatives and unbundled GLBEI funds and activities. The aggregate effect of these developments was that GLAP IV expenditures were no longer tagged or coded at the program level by Environment Canada partners. Accordingly, total expenditures and salary, as compared to project or contribution dollars, are not available for GLAP IV overall.

This particular cost-efficiency analysis can therefore be conducted only for the GLSF. However, within this program there have been changes as well, including a merger between the GLSF and the Restoration Program divisions and a reduction in overall contribution dollars, thus making expenditure analysis a challenge.

Based on currently available information (Table 3.4), GLSF project funding over four years totalled $12.66 million. A significant reduction in the annual funding amounts and number of projects funded annually after 2005‑2006 is noted. In total, 223 projects were funded, with the average cost per project being $57,478. With regards to cost-efficiency, for every contribution dollar an average of $0.24 is spent on salaries and O&M for the program. This is lower than the EcoAction Community Funding Program, which has a cost-efficiency ratio of $0.39 but is an intensive community-based delivery model. The GLSF ratio is higher than the Habitat Stewardship Program ($0.08) and the Invasive Alien Species Partnership Program ($0.13). The comparison with EcoAction is likely more appropriate given that GLAP IV staff also undertake significant community-based work (e.g., participation in RAP committees, partnership development).

Table 3.4: GLSF Cost-Efficiency Analysis*
 2005200620072008**

* Final expenditure figures for 2009‑2010 were not available at the time this report was being prepared.

** Estimate only, as the Great Lakes AOC Office and Remediation Program Division were merged in 2008‑2009.

O&M$196,000$120,000$120,000$162,500
Salaries$653,038$842,000$587,821$398,000*
Project Funding Total$5,423,936$2,566,283$2,624,950$2,047,015
No. of Projects95504038
Avg. Funding/Project$59,094$51,326$65,623$53,869
Avg. Admin+Salaries as % of Project Funding16%37%27%27%

Program managers and federal partners had favourable opinions about the efficiency of the program. When asked to elaborate on factors supporting program efficiency, interviewees noted the following:

  • GLSF leveraging. The file review indicates that, on average, GLSF funding constitutes 32 percent of overall project costs, consistent with program guidelines. Leveraging of volunteer efforts is also notable.
  • Continued directed call for GLSF proposals and sharpened focus on "essential-to-do" projects.
  • Science and monitoring work that is coordinated among provincial, state and federal counterparts.

On the other hand, respondents were also asked to discuss factors that undermine program efficiency. The following issues were identified:

  • The cumbersome nature of the initiative requires coordination efforts, due to the multiple jurisdictions, many players (within Environment Canada, and the number of OGDs), many AOCs and RAP committees, and a complex ecosystem.
  • Departmental transformation that occurred during the program created some disarray in the implementation of GLAP IV and consequent difficulties in monitoring the activities of federal partners and program financial expenditures.
  • A lack of a clear management mechanism for ongoing work planning and reporting.

Key informants' suggested means of improving cost‑efficiency focused on increasing communication (external and internal) and coordination to increase partner and public engagement in the restoration of AOCs, and to ensure that priority needs in the AOCs (remediation, science, monitoring and engagement) are cohesively linked to work planning. In addition, the ability to hire a stable core of staff was noted as a way to improve program efficiency by enhancing continuity of scientific/technical teams and reducing management time spent on human resources issues.

At the project level, federal and GLSF project proponents were asked about the extent to which their projects' activities and outputs were delivered in the most efficient manner. Respondents generally agreed that their projects were efficient, citing lean operations, collaborations, leveraging of funds from external partners and volunteer efforts. Any impediments to project efficiency were either described as events and constraints beyond the control of the proponents (e.g., staff turnover, weather interfering with fieldwork) or issues pertaining to the GLSF application and funding process. In the case of the latter, a few proponents stressed that constraints, such as the absence of multi‑year funding and the delays in funding approval, present challenges to carrying out projects at peak efficiency.

With respect to GLSF project implementation, the most frequent suggestion to improve cost-efficiency concerned the GLSF application process and approval times. This process was described as "lengthy" and, in the words of one respondent, involves "a great deal of red tape for both government and the proponent" and would require "streamlining the bureaucratic hurdles" in order to improve efficiency. Related to this, a few respondents pointed out that more could be accomplished within the same project (and thus greater efficiency could be achieved) if project approvals were better aligned with the seasonal nature of many projects and/or if agreements were multi-year. Other suggestions included: adopting a highly directive approach to funding GLSF projects to ensure that projects are tightly linked to delisting; and continuing to seek avenues to leverage funding (e.g., private donations, graduate student participation in projects).

Summary

The cost-efficiency of GLAP IV is difficult to determine with existing information, due to the departmental transformation that led to the unbundling of GLBEI funds and activities and a loss of program control over activities and funding. Program financial data is limited, as GLAP IV funds were not specifically coded at the program level.

Cost-efficiency analyses were conducted for the GLSF program component only. This analysis indicates that for every contribution dollar, $0.24 is spent on program administration (salary and O&M). This amount compares favourably to the EcoAction Community Funding Program, which has a cost-efficiency ratio of $0.39, but unfavourably with the Habitat Stewardship Program ($0.08) and Invasive Alien Species Partnership Program ($0.13). The GLSF is more comparable to EcoAction given both programs' more intensive involvement with project delivery (i.e., with EcoAction project proponents or with RAP committees for GLSF).

For key informants there is an impression of efficient program delivery both overall and at the project level. Factors identified as supporting program efficiency include: GLSF leveraging, continued directed calls for GLSF proposals, and coordination of science and monitoring activities with provincial and U.S. counterparts. Factors cited as undermining program efficiency refer to the complexity of the initiative (cross‑jurisdictional nature, number of partners and AOCs involved, and complexity of the lakes' ecosystems); the departmental transformation; and the lack of a clear management mechanism for ongoing work planning and reporting. At the project level, lean operations, collaborations, and leveraging of funds and volunteer efforts were viewed as contributing to efficiency, while impediments to efficiency arose from unanticipated events (staff turnover, weather) and inefficiencies in the GLSF application and funding process.

Top of Page

3.5 Performance: Program Economy

This section explores the extent to which GLAP IV is achieving its intended outcomes in an economical manner. The evaluation findings presented include the perceived cost‑effectiveness of GLAP IV and key informant opinions on alternative, more cost‑effective approaches to program delivery.

Overall Findings

There is consensus among key informants that GLAP IV is a good investment of public funds and that results are achieved in an economical way. Design aspects that are seen as contributing to the program's cost-effectiveness include: a focused effort on AOCs; cost‑sharing among jurisdictions and key stakeholders; local involvement and delivery; and continuity of intervention and knowledge acquisition over successive GLAP programs. A few key informants also noted the economic benefits of remediation, as well as the cost of not intervening.

No viable alternative approaches to achieve the same or better outcomes at a lower cost were identified. The prevailing sentiment among key informants is to "stay the course" in that GLAP provides a solid foundation for work in the AOCs. Fundamental changes to the program were discouraged by key informants as they may jeopardize the impact of work completed to date.

  • Evaluation Issue: Is GLAP IV achieving its intended outcomes in the most economical manner?
    • Are there alternative program models that would achieve the same expected outcomes at a lower cost?
  • Indicator(s):
    • Extent to which program's intended outcomes have been achieved at the lowest possible program cost
    • Views on whether good value is being obtained with respect to the use of public funds
    • Evidence of/views on whether there are alternative program models that would achieve the same expected outcomes at a lower cost
  • Methods:
    • Document review
    • Performance data analysis
    • File review
    • Key informant interviews
  • Rating: Achieved (~)

3.5.1 Cost-Effectiveness

The issue of economy concerns whether the achievement of program outcomes occurs in an economical manner and whether there are alternatives to the current program that could achieve the same outcomes at a lower cost. For GLAP IV, it is not possible to quantify total program expenditures due to the absence of financial tracking information. In addition, there is no program point of comparison to assess the relative effectiveness of the program in comparison to other approaches. The analysis is therefore based on key informants' subjective views on the cost-effectiveness of the program.

There was a consensus among key informants across all respondent groups that GLAP IV is a good investment of public funds. While there were some challenges in the implementation of the program (described previously), the majority opinion among key informants was that these challenges did not detract from the overall effectiveness and cost-effectiveness of the program.

Aspects of GLAP IV that were highlighted by key informants as contributing to cost‑effectiveness include the following:

  • Focused effort: investments in AOCs were perceived to maintain the focus of all levels of government on areas of joint priority, and the framework and science-based approach for selection and implementation of remediation work supports this targeted approach. This is particularly important when there are finite resources, so that available funds can be focused for maximum impact. GLAP IV's renewed emphasis on "essential-to-do" remedial as well as scientific and monitoring work to delist AOCs was also noted by several key informants as a positive development to increase effectiveness.
  • Sharing of costs: shared jurisdiction and provincial COA funds, as well as leveraged funding from external partners through the GLSF (e.g., municipal, industry, foundations), increased the positive impact of the federal investment.
  • Local involvement: GLAP IV's value further lies in fostering partnerships and collaborations among many partners, which leverages community resources (e.g., volunteers). As well, the RAP committee structure encourages local involvement and the tapping of resources "on the ground" which was perceived to increase effectiveness and efficiency.
  • Longevity: the program draws on scientific and technical expertise, from federal and provincial jurisdictions, that has been built over the course of successive GLAP programs. This level of continuity and knowledge was perceived to contribute to the overall effectiveness of current efforts.
  • Return on investment: noted by a small number of interviewees are the broader benefits of Great Lakes cleanup, such as for tourism and industry, which contributes to the overall value of the GLAP IV investment. An example of this is a benefits assessment of Randle Reef sediment remediation in Hamilton Harbour conducted for Environment Canada in 2006. The assessment identified a number of environmental, economic and social benefits of remediation and assigned quantified accumulated benefits to each beneficiary. The result was an estimated cumulative total benefit for the local area of $126 million during the period 2007 to 2032 with implementation of the Randle Reef project only, and $914 million with full implementation of all remediation projects (totals are approximate, to provide an indication of how results might be used in a social cost-benefit analysis).

Summary: There is consensus among key informants that GLAP IV is a good investment of public funds and that results are achieved in an economical way. Design aspects that are seen as contributing to the program's cost-effectiveness include: a focused effort on AOCs; cost-sharing among jurisdictions and key stakeholders; local involvement and delivery; and continuity of intervention and knowledge acquisition over successive GLAP programs. A few key informants also noted the economic benefits as well as the cost of not intervening.

Top of Page

3.5.2 Alternative Approaches

Key informants proposed few alternative approaches to the GLAP IV model that would achieve program outcomes at a lower cost. One potential alternative to GLAP IV is the program delivery model used in U.S. AOCs. In the United States, the federal jurisdiction transfers funds to the state level for program implementation and there is a greater reliance on contracted research conducted by universities. This model, while feasible in the Canadian context, was not viewed as desirable by the small number of key informants who discussed the option, including provincial representatives. The transfer of federal funds to other jurisdictions or external organizations was perceived to undermine control over the funding, as well as federal visibility. The continuity of scientific expertise and leadership of the federal government were also viewed as contributing greatly to the success of Canada's efforts in the AOCs. As well, a model that involves contracting-out aspects of work was perceived to increase, not reduce, costs.

Another view, not typically raised as an alternative per se but worth noting here, is the preference expressed by several key informants in various respondent groups to take a broader approach to addressing degradation in the Great Lakes beyond the narrow boundaries of the designated AOCs. These respondents believe there is a need to expand the scope of GLAP priorities (and resources) in order to increase responsiveness to nearshore or lake-wide issues and to promote a more holistic systems-wide approach to addressing remediation of the Great Lakes.

With finite resources, however, key informants were far more likely to suggest a stay the course approach, and generally approved of the GLAP model to restoring AOCs and meeting Canada's commitments under the GLWQA and COA. (Some design and delivery improvements, discussed above, were preferred over potential alternative approaches.) In defence of the current model, key informants noted that there is 20 years of experience with GLAP IV and its predecessors, which has built a foundation of expertise, partnerships and supporting mechanisms (e.g., the GLSF funding program and agreements with other jurisdictions) to move AOCs toward delisting. Moreover, many key informants believe that GLAP investments are now beginning to reach fruition, with several AOCs poised to be delisted or designated an area in recovery.

Summary: No viable options for alternative approaches to achieve the same outcomes at a lower cost were identified. The prevailing sentiment among key informants is to "stay the course" in that GLAP IV provides a solid foundation for work in the AOCs. Fundamental changes to the program are discouraged by key informants as they may jeopardize work completed to date.


[20] See Budget 2010, p.106 (www.budget.gc.ca/2010/pdf/budget-planbudgetaire-eng.pdf).

[21] Environment Canada and U.S. Environmental Protection Agency. 2009. State of the Great Lakes 2009, Highlights.   National Wildlife Federation. 2005. Prescription for Great Lakes Ecosystem Protection and Restoration – Avoiding the Tipping Point of Irreversible Changes.

[22] These priority areas include: contaminated sediment assessment; habitat restoration; municipal infrastructure improvements; assessments of ecosystem health; monitoring of water quality and ecosystem health improvements; and management of federal programs and coordination with other partners.

[23] Options included: government, water companies, large companies, citizens, farmers or non‑governmental organizations.

[24] According to the COA Annex, a) Canada and Ontario will co-lead the RAP process in the Toronto and Region, St. Mary's River, St. Clair River and Detroit River AOCs; b) Canada will lead the RAP process in the Thunder Bay, Hamilton Harbour, Port Hope and St. Lawrence River AOCs; and c) Ontario will lead the RAP process in the Nipigon Bay, Jackfish Bay, Peninsula Harbour, Spanish Harbour, Wheatley Harbour, Niagara River and Bay of Quinte AOCs.

[25] Environment Canada. 2009. Status Report on the Implementation of an Ecosystem Approach in Environment Canada.

[26] Environment Canada. 2008. 2007‑2008 Departmental Performance Report. Available at: www.tbs-sct.gc.ca/dpr-rmr/2007-2008/inst/doe/doe00-eng.asp

[27] Government of Canada. "Speech from the Throne". October 16, 2007. http://www.pm.gc.ca/eng/media.asp?id=1859

[28] Environment Canada. 2009. 2008‑2009 Report on Plans and Priorities. Available at: www.tbs-sct.gc.ca/rpp/2008-2009/inst/doe/doe00-eng.asp  

[29] http://greatlakesrestoration.us/. Accessed on November 25, 2010.

[30] The review committee assigned first priority to those AOCs with the greatest potential for delisting in the short to medium term (Group 1 AOCs). These Group 1 AOCs include: Thunder Bay, Nipigon Bay, Peninsula Harbour, Wheatley Harbour, St. Lawrence River, Bay of Quinte and St. Clair River. In the COA, Group 1 AOCs are Nipigon Bay, Jackfish Bay, Wheatley Harbour and St. Lawrence River (Cornwall)

[31] With the dissolution of the planning section in the former Great Lakes and Corporate Affairs Branch (concurrent with and as a result of Environment Canada transformation), there was no single focal point to lead the planning function.

[32] These included, in 2004 and 2006, Sharing experiences and a Habitat workshop in 2008.

[33] The department receives a share of its annual budget from temporary (as opposed to core) funding allocations from the Treasury Board in order to deliver programs for various specific purposes. While typically used in the intended manner, the Deputy Minister has the authority to realign this temporary funding to address departmental priorities. Just such a realignment occurred in 2005-06 and this process has come to be known within Environment Canada as "unbundling".

[34] OPGs were the basis of the new management structure in the department to promote integrated management and decision-making in the context of a clearer view of departmental results and strategic direction.

[35] Examples of outcome-related objectives include: rehabilitation of aquatic and riparian habitat resulting in re‑establishment of fish and wildlife; and monitoring and reducing rural non-point source pollution.

[36] Goal 2 in Annex 4 is to "Continue to improve the discovery and sharing of data, information and trends in the Great Lakes Basin Ecosystem" with the expected result of "Increased sharing of data and information among governments, organizations and Basin residents", including through web-based means.

[37] Treasury Board of Canada Secretariat. 2006. From Red Tape to Clear Results: The Report of the Independent Blue Ribbon Panel on Grant and Contribution Programs. Ottawa.

[38] Environment Canada. May 2009. Grants and Contributions Reform – Presentation to Executive Management Committee.

[39]   Interviews with federal partners and RAP committee members focused on success relating to specific program outcomes, while interviews with GLSF proponents, experts and committee members focused on success in more general terms.

[40] In 2005, there were 98 BUIs.

[41] In fact, Canada has renewed its commitment to GLAP in Budget 2010 for $8 million per year (http://www.budget.gc.ca/2010/pdf/budget-planbudgetaire-eng.pdf p106)

Return to Table of Contents

4.0 Conclusions

4.0 Conclusions

The evaluation of Environment Canada's GLAP IV focused on the five-year span of the program, from fiscal years 2005‑2006 to 2009‑2010. The findings of the evaluation lead to the following broad conclusions about the relevance, design and delivery, and performance (effectiveness, efficiency and economy) of GLAP IV.

4.1 Relevance

4.1.1 Continued Need for the Program

1) The evaluation evidence indicates that GLAP IV is an important program to address federal commitments under the GLWQA and COA with respect to Great Lakes AOCs. The program is needed to continue to address the complex and evolving Great Lakes ecosystem. Public opinion supports the continued need for the program, as Canadians indicate water resources to be a high priority. The work toward delisting of AOCs is not yet complete and, in light of finite resources, the use of the AOC designation is a way to focus resources and attention (with a science-based framework and including local involvement) on these highly degraded areas that impact the overall quality of the Great Lakes. The GLAP IV program priority areas continue to resonate and do not require adjustment.

2) GLAP IV is the only program that focuses on Great Lakes AOCs, acting as a catalyst for leveraged funding from other sources. While there are other sources of funding that support remediation as well as science and monitoring in the AOCs, there was no evidence of risk of duplication or overlap. This risk is mitigated by coordination at the program‑officer level and through collaborative networks of provincial, federal and U.S. scientists and technical experts.

4.1.2 Alignment with Federal and Departmental Priorities

3) GLAP IV aligns well with federal and departmental priorities. At the federal level, a "healthy environment for Canadians," including clean water, is a stated priority, and at the departmental level, GLAP IV is consistent with Environment Canada's mandate to preserve and enhance the quality of the natural environment. Moreover, the Great Lakes are identified as a high‑priority ecosystem nationally and for the Department.

4.1.3 Consistency with Federal Roles and Responsibilities

4) The shared boundary of the Great Lakes waters with the United States and Canada's commitments under the GLWQA provide a strong rationale for federal jurisdiction in the AOCs. As well, a number of legislative authorities (the International Boundary Waters Treaty Act, Clean Water Act and Canadian Environmental Protection Act, 1999) support federal jurisdiction in this area.

Top of Page

4.2 Program Performance: Design and Delivery

5) The GLAP IV model is a sensible approach to achieve program outcomes. Engagement of partners and involvement at the local AOC level are distinguishing and positive features of the model that leverage expertise and resources for remediation. The overarching framework for guiding GLAP IV and the science-based approach are also notable strengths that organize efforts in the AOCs and maintain the focus on delisting.

6) Delivery of some aspects of GLAP IV is inconsistent with the original design. The annual work planning process for the program and aspects of the program's management framework were not implemented. Therefore, few levers were available to the program to monitor and ensure that the activities to achieve the goals of GLAP IV, and the activities for which partners were funded, were undertaken in a coherent way. The distributed responsibility model was an identified weakness in other reviews of the program, with a recommendation to ensure that appropriate resource allocation, capacity and priority are assigned to meet program goals. Evidence suggests that many or all of these issues stemmed from the departmental transformation that occurred in 2005‑2006.

7) Resources for funding federal and GLSF partners to undertake priority activities in AOCs are largely sufficient, with increases needed to address inflation as well as the declines in funding to the GLSF that followed the first year of GLAP IV. The effectiveness and efficiency of departmental partners' efforts was attenuated by a limitation on the ability of Environment Canada programs to convert O&M to salary dollars, which created challenges in hiring the human resources required to achieve project objectives.

8) Financial accountability of the program was weak. Following transformation, program resources were no longer tracked by program but rather by departmental result. Furthermore, all B-base funding (i.e., temporary funding received for a specific purpose) was unbundled at this time, and so financial tracking of GLAP IV funding at the program level by internal Environment Canada partners was very limited. Some federal partners may not have uniformly received the GLAP IV funds that were allocated for their approved work plans.

9) Existing processes for reporting of GLSF-funded projects are quite complete (though only entered electronically to 2006‑07) and are well-understood by project proponents and managers. Reporting is focused to some extent on achievement of outputs (as opposed to outcomes, which are far more challenging to measure). For projects conducted by federal partners, processes for reporting to the program are not clearly articulated and have been sporadic over the life of the program.

10) Conducting monitoring and measuring results for program performance purposes is weak. Program reporting is not currently guided by a performance measurement framework and no systematic process exists to aggregate or summarize project-based reporting (costs [including partner contributions], activities and outputs).

11) The design and delivery of the GLSF component of GLAP IV is generally satisfactory. The program is longstanding and many proponents are involved in ongoing projects under this program. GLSF funds are an important catalyst in leveraging funding from other sources. The deficiencies in delivery of the program are the delays between proposal submission and funding approval (due in part to the program transition from the use of MOUs to contribution agreements as part of a Department-wide initiative to increase accountability), and the complexity of the application process. As several AOCs are moving closer to delisting, developing delisting criteria and clarifying responsibilities and the process for delisting are also emerging issues.

4.3 Program Performance: Achievement of Program Outcomes

12) GLAP IV has made solid progress in achieving shorter-term outcomes, but only modest progress toward long-term outcomes that build on the achievements of predecessor GLAP programs. Only three AOCs have been delisted and one AOC designated an area in recovery, and these occurred prior to GLAP IV. Moreover, a majority of the BUIs identified in 2003 remain impaired. The original program goal of completing federal actions in seven group 1 AOCs has not been achieved. Still, momentum appears to be increasing as the program expects that three group 1 AOCs will be delisted within two years. As well, a host of external factors influence program effectiveness, including the dynamic and complex nature of the ecosystem, the program's dependence on contributions of other partners to achieve its outcomes, and significant challenges in developing delisting criteria for BUIs that must be addressed.

13) Unintended impacts of the program are few in number but positive in nature, such as unanticipated public interest, engagement of partners and technical transfer.

4.4 Program Performance: Cost-Efficiency

14) Although data related to administrative expenses for the first four years of the program were available for the GLSF component only (with administrative costs for 2007‑2008 based only on program estimates), these limited data indicate that efficiency of this component compares reasonably well with other contribution programs at Environment Canada that require more intensive staff involvement in program delivery. There are, however, a number of avenues to improve efficiency at the program level (e.g., improve strategic management, and coordination among federal partners) and at the GLSF project level (e.g., streamlined application process).

4.5 Program Performance: Cost-Effectiveness

15) GLAP IV is an appropriate investment of public funds. Results are beginning to be achieved in the AOCs, though time frames for delisting have been long. As well, there are some broader economic benefits to restoration of AOCs. There were no alternative approaches that would result in a more economical achievement of program objectives.

Return to Table of Contents

5.0 Recommendations

5.0 Recommendations

Recommendations for the current or future iterations of the GLAP are based on the findings and conclusions of the evaluation. Overall, GLAP IV continued to be a relevant program to address ongoing needs for ecological restoration in the AOCs and is well‑aligned with departmental and federal priorities. At the end of its five-year term, GLAP IV has not fully achieved its intended longer-term outcomes in the areas of addressing BUIs and delisting AOCs, due in large part to the complexity and long-term nature of the task. The following recommendations focus on improvements to the management of the program in three areas: delivery and oversight; financial management and accountability; and performance measurement and reporting.

Delivery and Oversight

Oversight of GLAP IV was challenged by several changes that occurred within Environment Canada during the program's five‑year term. The changes affected both the overall management and the coordination of the program, and the GLSF. One recommendation pertains to improving coordination and accountability of the program, and one pertains to ensuring the timeliness and responsiveness of the GLSF.

1) The Regional Director General (RDG)‑Ontario should develop a suitable instrument to enhance coordination of federal partners involved in GLAP and the accountability of their efforts toward achieving GLAP goals. GLAP IV is a complex initiative that requires the contribution of many federal partners to achieve program goals. With the demise of the work plan review teams and process, GLAP IV lacked the instruments to ensure that these efforts were coordinated and that federal partners were implementing work plans to achieve program goals. The work planning process, convened annually as it was envisioned, could be re-instituted to enhance coordination and accountability.

2) The RDG‑Ontario should explore means of streamlining both the GLSF funding approvals process and the GLSF application process. While mostly satisfied with the GLSF program, proponents identified two issues with respect to administration: timeliness of approvals and an overly complex application process. Consideration should be given to scrutinizing the timing of the funding cycle to ensure that funding approvals coincide with seasonal activities, and that the application process is streamlined to promote efficiency (perhaps by exploring adoption/applicability of common application forms/guidelines being developed as part of the wider federal G&C reform exercise). Although most contribution agreements are single-year only, the majority of projects are of an ongoing nature. The use of multi-year agreements may be appropriate in some of these instances.

Financial Management and Accountability

There is limited information on GLAP IV program expenditures. Financial tracking of GLAP IV funds was weak during this iteration of the program due to limited use of codes for expenditures at the program level. One recommendation pertains to financial monitoring of GLAP funds at Environment Canada.

3) The RDG‑Ontario should explore ways to enhance the financial accountability of the program. The direct transfer of funds to internal partners and absence of program-specific coding of expenditures within the Department has weakened financial accountability of the program. Diligent coding of expenditures at the program level is a key element to ensure transferred funds are received and utilized within Environment Canada by departmental partners to meet program goals and priorities. The merit and potential disadvantages of direct transfers of program funds within Environment Canada to departmental partners for financial accountability should be examined.

Performance Measurement and Reporting

GLAP IV monitoring and reporting was not guided by a performance measurement framework. As a result, a broad spectrum of performance measures for federal partners and the GLSF were not identified early in the program and reporting requirements, particularly for federal partners, were not clearly articulated. Five recommendations pertain to improving performance measurement and reporting.

4) The RDG‑Ontario should develop a more regular and robust reporting approach for GLAP projects conducted by federal partners. Performance reporting for projects conducted by federal partners was unspecified and sporadic. A more robust reporting approach should be based on clear terms and conditions for reporting on funds allocated to federal partners. Reporting should link activities/deliverables to approved work plans, recognizing that, in some cases, federal projects are undertaken as part of an integrated science program.

5) The RDG‑Ontario should develop a performance measurement framework for the program, and include a spectrum of more sensitive performance measures (shorter- and longer-term). The program does not have a performance measurement framework and is not currently well‑positioned to demonstrate progress and interim results because measures such as delisting AOCs are long-term. A more nuanced approach must balance the benefit of performance measurement with the cost of monitoring and assessing BUIs in the AOCs. Recent efforts by the program to document accomplishments and assign roles, responsibilities and timelines for outstanding actions and priorities are acknowledged as a strong foundation for this ongoing work.

6) The RDG‑Ontario should support the development of information management tools to enhance implementation and monitoring of GLSF and federal projects. No systematic process exists for monitoring and reporting the activities, outputs and performance for GLAP-funded projects or the program overall. A Department-wide Management Information System for Environment Canada's community funding programs, including the GLSF, is in development, although this system is not expected to be implemented until 2012. Opportunities for interim tracking of activities and project and partner contributions should be identified.

7) The RDG‑Ontario should support the development of information-sharing tools to facilitate broader access to the research generated by GLAP IV by the scientific community and the public. There is no ongoing mechanism available for technical transfer of research and results across the AOCs. This is an outstanding commitment in Annex 4 of the COA.

8) The RDG‑Ontario should continue efforts to define delisting criteria for AOCs and clarify responsibilities with respect to delisting decision making. Roles and responsibilities for aspects of delisting AOCs are a grey area and may be variable across the AOCs. As more AOCs approach restoration, clarifying the steps to delisting will increase in importance.

Return to Table of Contents

6.0 Management Response

6.0 Management Response

This section outlines the management response to the evaluation recommendations. The RDG‑Ontario accepts the evaluation and all of its recommendations, and the Great Lakes Division has provided a plan to implement the following management actions in response to the evaluation recommendations within the context of the program's renewal.

1. The RDG‑Ontario should develop a suitable instrument to enhance coordination of federal partners involved in the GLAP and the accountability of their efforts toward achieving GLAPgoals.GLAP IV is a complex initiative that requires the contribution of many federal partners to achieve program goals. With the demise of the work plan review teams and process, GLAP IV lacked the instruments to ensure that these efforts were coordinated and that federal partners were implementing work plans to achieve program goals. The work planning process, convened annually as it was envisioned, could be re-instituted to enhance coordination and accountability.

The RDG‑Ontario agrees with this recommendation.

A GLAPfive-year work plan will be developed by Environment Canada, Great Lakes Division, with input from all federal departments and Responsibility Centres within Environment Canada contributing to the restoration of Great Lakes AOCs and accessing GLAP funds. The GLAP Work Plan Review Team will be re-established and will annually conduct a review of progress achieved, identify actions required to complete the restoration of AOCs, determine priorities for the coming year, and propose adjustment of the work plan and funding allocations as required. The GLAP Work Plan Review Team will be led by the Great Lakes Division of Environment Canada and will comprise representatives of Environment Canada, the Department of Fisheries and Oceans, Public Works and Government Services Canada and other federal government departments as necessary. Recommendations of the GLAP Work Plan Review Team will be presented to the Director, Great Lakes Division, for approval. A GLAP work planning template will be established specifying the project rationale, (i.e., how the proposed project contributes to AOC delisting), scope of work, deliverables, schedule and resource requirements.

Timeline

Deliverable(s)

Responsible Party

August 2010

GLAP Work Plan Review Team established

Director, Great Lakes Division

October 2010

Establish GLAP work planning and reporting template

Director, Great Lakes Division

November 2010

Five-year work plans for all federal departments and groups within Environment Canada accessing GLAP funds contributing to the remediation of AOCs

Director, Great Lakes Division

March‑April 2011, 2012, 2013, 2015

Review and revision of GLAP Work Plan, Annual Reports on Results, funding allocation decision

Director, Great Lakes Division; and GLAP‑funded groups

Top of Page

2. The RDG‑Ontario should explore means of streamlining both the GLSF funding approvals process and the GLSF application process. While mostly satisfied with the GLSF program, proponents identified two issues with respect to administration: timeliness of approvals and an overly complex application process. Consideration should be given to scrutinizing the timing of the funding cycle to ensure that funding approvals coincide with seasonal activities, and that the application process is streamlined to promote efficiency (perhaps by exploring adoption/applicability of common application forms/guidelines being developed as part of the wider federal G&C reform exercise). Although most contribution agreements are single-year only, the majority of projects are of an ongoing nature. The use of multi-year agreements may be appropriate in some of these instances.

The RDG‑Ontario agrees with this recommendation.

Environment Canada's Action Plan to Reform the Administration of Grants and Contributions will improve the efficiency of the GLSFapplication and approval process.

In addition, improvements have been made to the GLSF application and proposal evaluation processes to simplify and improve the efficiency and consistency of the GLSF application and approval process. Multi-year agreements will be used where appropriate.

 

Timeline

Deliverable(s)

Responsible Party

June 2010

Adopt application and reporting processes from the Departmental Action Plan to Reform the Administration of Grants and Contributions

Director, Great Lakes Division

Top of Page

3. The RDG‑Ontario should explore ways to enhance the financial accountability of the program. The direct transfer of funds to internal partners and absence of program-specific coding of expenditures within the Department has weakened financial accountability of the program. Diligent coding of expenditures at the program level is a key element to ensure transferred funds are received and utilized within Environment Canada by departmental partners to meet program goals and priorities. The merit and potential disadvantages of direct transfers of program funds within Environment Canada to departmental partners for financial accountability should be examined.

The RDG‑Ontario agrees with this recommendation.

The Great Lakes Division will take steps to ensure that departmental resources expended in the delivery of the GLAP are adequately documented. To this end, the Great Lakes Division will take the necessary steps to adopt program‑specific authority codes for all Environment Canada internal salary, O&M and G&C expenditures of GLAP funds. Furthermore, all GLAP funds will be managed by the Great Lakes Division and distributed to other federal departments and Responsibility Centres within Environment Canada in accordance with the GLAP five‑year work plan annual updates.

Timeline

Deliverable(s)

Responsible Party

May 2010 (done)

Request for the establishment of a GLAP‑specific authority code in the Financial Information System

Director, Great Lakes Division

May 2010 (done)

Establish program‑specific authority codes for GLAP salary, O&M and G&C funds

Environment Canada Finance Directorate

Top of Page

4. The RDG‑Ontario should develop a more regular and robust reporting approach for GLAP projects conducted by federal partners. Performance reporting for projects conducted by federal partners was unspecified and sporadic. A more robust reporting approach should be based on clear terms and conditions for reporting on funds allocated to federal partners. Reporting should link activities/deliverables to approved work plans, recognizing that, in some cases, federal projects are undertaken as part of an integrated science program.

The RDG‑Ontario agrees with this recommendation.

Environment Canada, Great Lakes Division, will require annual reports on all work plan activities from Environment Canada and other federal partners participating in the restoration of Great Lakes AOCs. Reporting information will be considered in the annual review and revision of work plans. Clear guidelines will be established for monitoring and reporting of GLAP funds by Environment Canada and federal partners. This will include the establishment of a short reporting template, including financial reporting, and pertinent performance information.

Timeline

Deliverable(s)

Responsible Party

October 2010

Establish guidelines for monitoring and reporting of GLAP funds; GLAP work reporting template established

Director, Great Lakes Division

March‑April 2011, 2012, 2013, 2015

An annual report on the status of GLAP work plan commitments will be prepared and reviewed through the annual GLAP work planning process

Director, Great Lakes Division

Top of Page

5. The RDG‑Ontario should develop a performance measurement framework for the program, and include a spectrum of more sensitive performance measures (shorter- and longer-term). The program does not have a performance measurement framework and is not currently well‑positioned to demonstrate progress and interim results because measures such as delisting AOCs are long-term. A more nuanced approach must balance the benefit of performance measurement with the cost of monitoring and assessing BUIs in the AOCs. Recent efforts by the program to document accomplishments and assign roles, responsibilities and timelines for outstanding actions and priorities are acknowledged as a strong foundation for this ongoing work.

The RDG‑Ontario agrees with this recommendation.

A performance measurement framework will be developed that will consider feasible measures to provide a more detailed and incremental assessment of progress toward restoration of BUIs and delisting of AOCs. An Assessment of the Status of Remaining Actions to Delist AOCs has been completed and will serve as an activity‑based, short‑term performance measurement framework. A Beneficial Use Impairment Status and Progress Report has also been completed and provides a longer‑term, result‑based, performance measurement framework. The Assessment and Report will be reviewed, revised and reported on biennially in alternating years.

Timeline

Deliverable

Responsible Party

March 2011

Performance measurement framework developed for the program

Director, Great Lakes Division

February 2012, February 2014

Updated Beneficial Use Impairment Status and Progress Report

Director, Great Lakes Division

February 2011, February 2013, February 2015

Updated Assessment of the Status of Remaining Actions to Delist AOCs

Director, Great Lakes Division

Top of Page

6. The RDG‑Ontario should support the development of information management tools to enhance implementation and monitoring of GLSF and federal projects. No systematic process exists for monitoring and reporting the activities, outputs and performance for GLAP-funded projects or the program overall. A Department-wide Management Information System for Environment Canada's community funding programs, including the GLSF, is in development, although this system is not expected to be implemented until 2012. Opportunities for interim tracking of activities and project and partner contributions should be identified.

The RDG‑Ontario agrees with this recommendation.

Environment Canada's Action Plan to Reform the Administration of Grants and Contributions (the Department's G&C reform initiative) is developing an online application and information management system for G&Cprograms. This system will improve program efficiency, enhance alignment with departmental priorities and improve the ability to report collectively on the results of departmental funding programs. The system is scheduled to be in operation by 2011‑2012. In the interim, the Great Lakes Division has established an Excel‑based system to track GLSF project proposals, proposal reviews and selection, project recipients and contribution agreements, and project financials, products and outcomes.

Timeline

Deliverable

Responsible Party

2010

Adopt information management system from the Departmental Action Plan to Reform the Administration of Grants and Contributions.

Director, Great Lakes Division

Top of Page

7. The RDG‑Ontario should support the development of information-sharing tools to facilitate broader access to the research generated by GLAP IV by the scientific community and the public.There is no ongoing mechanism available for technical transfer of research and results across the AOCs. This is an outstanding commitment in Annex 4 of the COA.

The RDG‑Ontario agrees with this recommendation.

Information on AOCs is provided through the Environment Canada website and other communications products. Research findings are regularly published in scientific journals and presented at scientific forums, including the Great Lakes State of the Lakes Ecosystem Conference and the International Association for Great Lakes Research Conference.

Additionally, AOCProgress Reports have been developed jointly with the Province of Ontario for release in 2010. This tool will be built upon and modified to publicly report on progress regarding remediation of Great Lakes AOCs, and to enhance sharing of research results and other information. Reporting on AOCs will be implemented on a three‑year cycle, consistent with other Great Lakes reporting.

Timeline

Deliverable

Responsible Party

2013

Area of Concern Progress Reports will be issued every three years and will communicate research results to the scientific community and general public

Director, Great Lakes Division

Top of Page

8. The RDG‑Ontario should continue efforts to define delisting criteria for AOCs and clarify responsibilities with respect to delisting decision making. Roles and responsibilities for aspects of delisting AOCs are a grey area and may be variable across the AOCs. As more AOCs approach restoration, clarifying the steps to delisting will increase in importance.

The RDG‑Ontario agrees with this recommendation.

The GLWQA clearly states that the parties (Canada and the United States) are responsible for leading all aspects of the AOC process "in cooperation with State and Provincial Governments."

Environment Canada's January 2010 Great Lakes RAP Workshop, which included federal and provincial representatives involved in Great Lakes AOCs as well as the local RAP coordinators, addressed the delisting process issue as an agenda item. The principle outcome was a commitment to develop a Canada‑Ontario Guide to the Designation of Beneficial Use Impairments, Preparing Stage 3 Reports and Delisting Areas of Concern. The guide, which is being developed collaboratively with the Ontario Ministry of the Environment, will clarify the responsibilities with respect to delisting decision making.

Timeline

Deliverable

Responsible Party

December 2010

Canada‑Ontario Guide to the Designation of Beneficial Use Impairments, Preparing Stage 3 Reports and Delisting Areas of Concern

Director, Great Lakes Division

Return to Table of Contents

Table 1.2

Evaluation of the Great Lakes Action Plan IV

Final Report: June 10, 2010

Table 1.2: Number and Value of Federal GLAP IV Projects Recommended for Funding by Priority Area and Partner
GLAP IV Priority AreaNWRIECBDFOPWGSCTotal
No. of ProjectsApproved FundingNo. of ProjectsApproved FundingNo. of ProjectsApproved FundingNo. of ProjectsApproved Funding
Source: Environment Canada. GLAP IV Total Work Plan, August 21, 2008.xls. Internal document.
Habitat Restoration----1$7,169,0002$220,000----$7,389,000
BUI Assessments4$2,665,0003$3,375,00014$4,223,000----$10,263,000
Monitoring1$250,0001$604,000--------$854,000
Municipal Infrastructure2$422,0001$5,080,000--------$5,502,000
Contaminated Sediment Assessment and Remediation1$150,0001$12,871,000----4$650,000$13,671,000
Coordinating Actions among Partners----2$1,626,000--------$1,626,000
Total8$3,487,0009$30,725,00016$4,443,0004$650,000$39,305,000

Return to Table of Contents

Table 1.3

Final Report: June 10, 2010

Table 1.3: Value and Number of GLSF Contribution Agreements, 2005-2010
  Group 1 Group 2 Group 1 & 2 (multiple AOCs)/ No group identified Total
Number Value Number Value Number Value Number Value
< $25,000 30 $363,732 49 $706,017 7 $54,003 86 $1,137,062
$25,000-$74,900 33 $1,431,265 71 $3,148,331 5 $125,390 109 $4,704,986
$75,000+ 32 $3,953,542 38 $5,809,144 2 $207,000 72 $9,969,686
Total 95 $5,748,539 158 $9,663,492 14 $424,703 267 $15,836,734

Return to Table of Contents

Figure 1

Final Report: June 10, 2010

Figure 1: Logic Model: GLAP IV

This figure presents a graphical depiction of program of the Great Lakes Action Plan IV.

This figure shows the program logic model, which presents a graphical depiction of how the activities and outputs of the Great Lakes Action Plan IV relate to immediate, intermediate, long-term and ultimate outcomes.

Return to Table of Contents

Date modified: