Final Report: June 10, 2010
Previous page | Table of Contents | Next page
This section presents the evaluation findings on the continued need for GLAP IV, the alignment of the program with federal and departmental priorities, and the extent to which GLAP IV is consistent with federal roles and responsibilities.
The GLAP remains a relevant program that enables the federal government to address the continued need for restoration and maintenance of the Great Lakes AOCs. While recent trends in Great Lakes' ecosystem conditions are variable, historical sources of stress and new challenges are generating negative impacts in many areas of the lakes. Environmental need, together with the social and economic benefits of the Great Lakes, supports the continued relevance of the program. Public opinion is consistent with scientific information and key informant views on the environmental and societal importance of the Great Lakes and the continued need for the GLAP.
There is little redundancy risk associated with GLAP IV. The program uniquely addresses federal commitments in the GLWQA and the COA by providing a framework to address BUIs linked to AOC RAP priority areas. Unlike other public funding (federal, provincial and municipal) and non-public funding (industry, foundations, non‑governmental organizations, etc.), GLAP IV is targeted exclusively to AOCs. Key informants express few concerns about the possibility of duplication or overlap between GLAP IV and other programming. Rather, GLAP IV is seen as highly complementary and a catalyst in orienting other funding sources to provide support in the AOCs.
GLAP IV is consistent with federal and departmental priorities. The program is part of the GLBEI, one of six ecosystem initiatives at Environment Canada under the Priority Ecosystem Initiatives Sub-Activity. The GLBEI supports the Department's strategic outcome that "Canada's natural capital is restored, conserved and enhanced." GLAP IV outcomes and priority investments also support a number of other sub-activities within Environment Canada's program activity architecture, including Aquatic Ecosystems, Wildlife, Priority Ecosystems, Ecosystems Sustainability, and Assessment and Ecological Monitoring.
Environment Canada's GLAP IV is aligned to support federal government commitments and obligations under the GLWQA and the COA, as well as federal government priorities outlined in the 2007 Speech from the Throne and two recent federal budgets. Other legislative authorities include the Canada Water Act, the International Boundary Waters Treaty Act, and the Canadian Environmental Protection Act, 1999.
For over four decades, the GLWQA and COA have served as important and effective mechanisms for protecting and restoring the Great Lakes. Canadian and United States actions implemented under the GLWQA have resulted in the lowering of phosphorus loads to the Great Lakes, and improvements in several water quality indicators, in particular in the more heavily nutrient‑impacted lower lakes, as indicated in the State of the Lakes Ecosystem Conference's State of the Great Lakes 2009 report. At the same time, a 2007 review of the GLWQA by the IJC found that stronger linkages are needed between the Agreement's stated overall purpose – to restore and maintain the chemical, physical and biological integrity of the waters of the Great Lakes Basin ecosystem – and the measures outlined in the Agreement's articles and annexes. The review further emphasized that the Agreement should be revised to address today's pressing issues, including the impacts of climate change, aquatic invasive species and urbanization. There was also recognition that these issues affect biodiversity. Accordingly, in June 2009 the federal governments of Canada and the United States announced a commitment to strengthen and modernize the Agreement so as to better address pollution, invasive species and climate change.[20]
Recent trends of Great Lakes ecosystem conditions vary: some conditions are improving and some are deteriorating.[21] Efforts under the GLWQA directed at degradation and contamination in AOCs through the implementation of RAPs and lake‑wide management plans have coordinated activities directed at addressing numerous BUIs. Yet, despite the progress made to date, the Great Lakes are exhibiting symptoms of extreme stress from a combination of sources, including toxic contaminants, invasive species, nutrient loading, shoreline and upland land use changes, and hydrologic modifications. In large areas of the lakes, historical sources of stress have combined with new ones to reach tipping points, at which ecosystem-level changes occur rapidly and unexpectedly, confounding traditional relationships between sources of stress and expected ecosystem responses. In a 2008 TB MAF review of the GLBEI, an examination of the research concluded that, despite significant progress and many successes in addressing past challenges, the Great Lakes continue to be at risk of unprecedented changes due to a range of stresses acting in combination, including: degradation of the nearshore zone; continued introduction and spread of aquatic invasive species; significant changes to the lower food web; wetland and other natural habitat loss; chemical contamination; threats to drinking water; and impacts of climate change.
Among key informants, there is consensus across all respondent groups on the continued need for GLAP IV. Almost all interviewees agree that there is a need for federal funding to support the restoration and maintenance of Great Lakes AOCs. The Great Lakes are a priority ecosystem with strong economic and social importance for all Canadians. For most interviewees, the restoration and maintenance of the Great Lakes represents not only a shared responsibility among all jurisdictions (federal, provincial and municipal), but one for which federal leadership and funding is critical for a number of reasons (among them, the federal mandate for transboundary waters, in accordance with the Canada Water Act and the International Boundary Waters Treaty Act).
A majority of interviewed key informants indicate that there is a continued need for federal funding to ensure the coordination and leveraging of funding with other jurisdictions and stakeholders to restore and maintain the Great Lakes. A minority of interviewees (though a majority of committee members and external experts) refer to the federal government's jurisdictional responsibilities for international waters as well as those outlined in international and national agreements, as the rationale for the continued need for GLAP IV. A number of interviewees (including a majority of committee members and of federal managers and partners) cite the magnitude of the remaining issues and problems facing the Great Lakes in general, and the AOCs in particular, as evidence for the continued need for federal leadership and funding. For a few GLSF project proponents, the issue of "legacy sites" and non-point source contaminants, and the broader economic benefits of the lakes as a resource, represent additional reasons for the continued need for federal funding in support of the Great Lakes AOCs.
While the AOC designation is useful in focusing attention and resources on areas of stress, some key informant interviewees indicate there is also a need for a more holistic lake- or ecosystem-wide approach to adequately address environmental issues affecting the health of the Great Lakes ecosystems. Some respondents believe there is a need to expand the focus of GLAP IV beyond the designated AOCs. A few respondents indicate that the GLAP should be broadened to include activities identified in the lake-wide management plans developed for each of the Great Lakes. For a small number of interviewees, there is a perceived need to broaden the focus to include shoreline and watershed remediation activities considered to be important for the long‑term integrity and sustainability of the Great Lakes.
Most key informants believe all GLAP IV priority investment areas are important and that there is an ongoing need to continue supporting all targeted priority areas.[22] A majority of interviewees underscore the need for Environment Canada to be flexible in maintaining the GLAP's responsiveness to the different needs of the individual AOCs (i.e., that priorities need to be assessed on an AOC by AOC basis).
Public opinion is consistent with scientific and key informant views on the importance of the Great Lakes and the continued need for support. Recent public opinion research reveals a growing concern among Canadians about Canada's freshwater resources and the need for governments to ensure the availability of freshwater supplies. A 2009 Nanos poll found that six in ten Canadians (62 percent) consider freshwater to be our most important natural resource, compared to two out of ten (22 percent) who feel oil and gas are the most important. Also, four in ten Canadians (40 per cent) indicate that water pollution from industry, agriculture and/or households is their "greatest concern" regarding freshwater in Canada.
Canadians' confidence in the supply and safety of our freshwater resources has declined significantly in recent years. The 2009 Water Attitudes Study found that, while nearly all Canadians believe it is important to conserve freshwater on an ongoing basis (95 percent), confidence that Canada has enough freshwater for the long term has declined by 11 percentage points, from 81 per cent in 2008 to 70 per cent in 2009. Similarly, Canadians' confidence in the safety of Canada's freshwater supply has declined, from 81 per cent in 2008 to 72 per cent in 2009. Seven in ten Canadians (70 per cent) believe that government is the "most responsible" for ensuring the availability of clean water for their communities.[23]
The Great Lakes United Poll (2007) found that Ontarians are "very concerned" about untreated sewage entering the Great Lakes (82 per cent), contamination of the food chain by toxic pollution such as mercury (78 per cent), and the loss of wetlands and animal habitat (67 per cent). The poll also revealed that eight in ten Ontarians (78 per cent) favour spending $2 billion per year for ten years to restore the health of the Great Lakes and St. Lawrence River. Majorities of Ontarians agree that the following constitute "very good" reasons for spending Canadian tax dollars to clean up the Great Lakes: 10 million Canadians get their drinking water from the Great Lakes (83 per cent of Ontarians consider this a "very good" reason); we should act now because problems in the lakes will cost more to fix in the future (77 per cent); the Great Lakes represent one fifth of the world's freshwater supply (77 per cent); and contamination of the food chain (72 per cent). Ontarians also perceive a number of benefits to protecting the integrity of the Great Lakes, as evidenced by majorities of Ontarians who "agree" that: we all have a personal responsibility to leave the Great Lakes in good health for our children (99 per cent); government spending to upgrade sewage and water systems in the province will create jobs (89 per cent); and the Great Lakes will help shield us from the impacts of global warming like heat and drought (72 per cent).
Program documents and key informants point to the availability of several other funding programs that support remediation or science and monitoring in the Great Lakes AOCs. A key source is provincial funding for implementation of Ontario commitments under the COA. Relevant provincial ministries, the MNR and MOE, use these funds to fulfill provincial commitments in the COA, including those in Annex 1 pertaining to AOCs. In addition to provincial COA funds, funding sources include:
Unlike other funding sources, however, GLAP IV is uniquely targeted to remediation work in the Great Lakes AOCs. Also distinguishing GLAP IV from other initiatives is the program's framework for funding that is closely tied to Canada's stated AOC commitments under the GLWQA, priorities identified in the RAP reports in each AOC, and its science-based focus. Funding and work planning is thus linked to BUIs identified in each AOC, with the objective of completing federal actions to address BUIs in order to delist.
Key informants across all respondent groupings expressed few concerns about duplication or overlap between GLAP IV and other funding sources. Mitigating the risk of duplication between GLSF-funded projects and other funding sources is informal information-sharing at the officer level – for example, sharing of GLSF project proposal and funding lists among officers with responsibilities for the various funding programs. Also, more formally, the technical review of project proposals may also bring to light potential areas of funding overlap or areas where efficiencies among projects can be achieved. While GLSF project proponents also perceived GLSF and other funding sources to be complementary rather than overlapping, they noted that there was duplication of effort for project proponents in terms of their efforts to access funds from multiple sources (i.e., preparing funding applications for various funding programs).
The risk of duplication with respect to funding of science and monitoring activities is mitigated because work in the AOCs is conducted by a relatively small science community, with effective collaboration and awareness of each other's activities between federal researchers and their provincial and U.S. counterparts. As well, each AOC has an assigned federal or provincial lead, which further ensures that duplication and overlap is avoided.[24]
Summary: Recent trends of Great Lakes ecosystem conditions vary; some conditions are improving and some are deteriorating. In large areas of the lakes, historical sources of stress have combined with new ones to reach a tipping point, at which radical ecosystem-level changes can occur rapidly and unexpectedly. Public opinion is consistent with scientific and key informant views on the environmental and societal importance of the Great Lakes and the continued need for the GLAP. GLAP IV is targeted exclusively to AOCs, unlike other public (federal, provincial and municipal) and non-public (industry, foundations, NGOs, etc.) initiatives. Key informants express few concerns about the possibility of duplication or overlap between GLAP IV and other programming. Rather, GLAP IV is seen as complementary in that it is a catalyst for orienting other funding sources to support activities in the AOCs.
Environment Canada's GLAP is aligned to support federal government priorities established under the 2007 Speech from the Throne and two recent federal budgets. Notably, one of the five priorities set out in the October 2007 Speech from the Throne is "A Healthy Environment for Canadians." The Throne Speech commits the Government of Canada to implementing a new "water strategy" to help clean up major lakes and oceans. Budget 2007 created the Action Plan for Clean Water, which allocated funds to improve the quality of water in Canada's rivers, lakes and oceans, including $48.9 million over eight years to accelerate existing actions for the remediation of contaminated sediment in eight Great Lakes AOCs.
The Great Lakes are among the high-priority ecosystems identified nationally and recommended for coordinated action.[25] This recommendation was identified through the 2007‑2008 Priority Ecosystem Initiatives Management Framework initiative, which was intended to improve the Priority Ecosystems Initiatives program, optimize integration of Environment Canada programs and activities, and strengthen accountability and reporting. Ongoing federal support is further evident in Canada's signing in 2007‑2008 of a new three-year COA to work toward a healthy and clean Great Lakes Basin Ecosystem.[26]
Almost all key informants concurred that the objectives of GLAP IV are well-aligned with current federal government priorities, pointing most often to the GLWQA. By signing this Agreement, Canada has an obligation to work toward improving the Great Lakes. Some interviewees further point to the federal interest in water quality (relating to the 2007 Throne Speech, and Budget 2007 establishment of an Action Plan for Clean Water and a priority on providing "A Healthy Environment for Canadians"[27]) as evidence of GLAP IV alignment with federal priorities. In addition, some key informants (both internal departmental managers and partners, and external experts and RAP representatives) note that GLAP IV is an important vehicle to fulfill international commitments and to contribute to positive Canada‑U.S. relations.
The GLBEI is one of six ecosystem initiatives at Environment Canada under the Priority Ecosystem Initiatives Sub-Activity, and aligns with one of the department's seven program priorities (Ensuring Water Quality and Quantity).[28] The Priority Ecosystem Initiatives Sub‑Activity adopts an ecosystem approach to environmental management for the benefit of governments, industry and individuals, by aligning science, monitoring, on‑the‑ground action and policy expertise, as well as enhancing collaborative governance and decision-making mechanisms. The GLBEI supports the Department's strategic outcome that "Canada's natural capital is restored, conserved, and enhanced." GLAP IV outcomes and priority investments were designed to complement a number of other sub-activities within Environment Canada's program activity architecture, including Aquatic Ecosystems, Wildlife, and Assessment and Ecological Monitoring.
GLAP IV is considered by most key informants to be well-aligned with Environment Canada departmental priorities as a priority ecosystem. A number of federal respondents also note that activities within the Great Lakes, such as science and monitoring, have national applicability and that this research can be used to benefit areas beyond the Great Lakes. Furthermore, GLAP IV is viewed by key informants as a consistent fit with Environment Canada's mandate to preserve and enhance the quality of the natural environment (habitat, biodiversity, species at risk) and conserve and protect Canada's water resources.
Summary: GLAP IV aligns well with federal and departmental priorities. The program uniquely addresses federal commitments in AOCs under the GLWQA and COA, by providing a framework to address BUIs linked to AOC RAP priority areas. The GLBEI, of which GLAP IV is a part, is one of six ecosystem initiatives at Environment Canada under the Priority Ecosystem Initiatives Sub-Activity. The GLBEI supports the Department's strategic outcome that "Canada's natural capital is restored, conserved, and enhanced." GLAP IV outcomes and priority investments support a number of sub‑activities within Environment Canada's program activity architect, including Aquatic Ecosystems, Wildlife, and Assessment and Ecological Monitoring.
The Government of Canada, through the GLWQA, has committed to working with other levels of government to assess, restore and protect beneficial uses in AOCs. Legislative authorities include:
Key informants across all respondent groups concur that it is appropriate for the federal government to play a role in the restoration and maintenance of the Great Lakes AOCs. Collectively, key informants point to several factors that illustrate the need for a federal role in the Great Lakes. First, the federal government has a responsibility to support work to fulfill international agreements (such as the Great Lakes Water Quality Agreement). Further, as the Great Lakes are the largest freshwater body in North America, it is considered by the majority of key informants to be a critical national resource. For example, one key informant stated that "they are clearly important to water quality, industry and transportation, and affect a significant proportion of the Canadian population."
The complexity of the boundaries of the waters – four of five lakes form part of the Canada-U.S. border and five AOCs are binational – underscores the appropriateness of a federal lead in this area. An external expert summarized this point by noting that "because we are dealing with binational issues, federal funding that enables place-based remediation and protection to happen is paramount to the success of the government's promise to the [Great Lakes Water Quality] Agreement."
Some key informants feel that fulfilling this commitment is especially important now that the Government of the United States has committed $475 million in new funding to the Great Lakes under the U.S. Great Lakes Restoration Initiative.[29] This increase in funding is perceived by some key informants to raise expectations for Canada to provide a commensurate investment in the Great Lakes. Through GLAP IV efforts, some key informants point out that Canada has taken a leadership role relative to the United States on Great Lakes AOC restoration and maintenance. They urged that these activities, along with the financial commitment, continue.
Although key informants agree that there is a need for federal involvement, including a leadership role, it was also noted that the federal government should not and cannot play the exclusive role, and that provincial and municipal governments, as well as community-driven organizations, have a responsibility and critical role to play as well. An example of this is that the jurisdiction for wastewater infrastructure improvements rests with municipalities and the provinces.
Summary: The federal role in GLAP IV is appropriate. GLAP IV supports federal government commitments and obligations under the GLWQA and COA, as well as federal government priorities outlined in the 2007 Speech from the Throne and two recent budgets. Other legislative authorities include the Canada Water Act, International Boundary Waters Treaty Act, and Canadian Environmental Protection Act, 1999.
This section presents the evaluation findings related to the adequacy of the program design and delivery from a number of aspects outlined in the evaluation matrix, including the engagement of partners, work planning activities, roles and responsibilities, resource allocations, monitoring and reporting activities, and GLSF program delivery.
Overall, GLAP IV was viewed as a sensible model to achieve intended outcomes. Engagement of partners represents a critical and beneficial aspect of the GLAP IV program delivery. Partnerships occur at many levels and in many ways. Formal agreements between Canada and the United States (GLWQA) and Canada and Ontario (COA) represent the foundation for commitments of the jurisdictional parties with respect to the AOCs. Implementation involves horizontal partnerships within Environment Canada and across the federal government. RAP committees provide a forum for governments (federal, provincial, municipal and First Nations) and key stakeholder groups (conservation authorities, community groups and NGOs) to coordinate their efforts through to the delisting process. The funding structure of GLSF projects promotes partner engagement at the local and regional level with partner contributions, including cash and in-kind funding, assistance with delivery, and participation in advisory or research capacities.
The implementation of GLAP IV diverged from the original design of the program in several areas. The five-year work plans required by federal partner departments to access GLAP IV funds were considered to be a positive and worthwhile exercise. The intended work‑planning annual reporting, review and updating process did not occur as intended, however, which resulted in a loss of flexibility, responsiveness and accountability.
Due to the horizontal nature of the initiative and broad changes at Environment Canada, GLAP IV governance and management activities do not reflect those outlined in the management framework developed for the program. Areas identified as a potential source of confusion or concern by federal key informants include an inadequate instrument to ensure that partner contributions to program goals are fulfilled, and clarity of roles and responsibilities with respect to delisting AOCs.
For federal partners who received their recommended funding, the amount of GLAP IV funding was seen to be adequate, with the caveat that they would like future funding to be indexed for inflation. However, some federal partners report receiving significantly fewer resources than their original approved funding levels, which they feel negatively affected their ability to implement planned activities. For federal partners that received GLAP IV funding allocations in full, a more pressing concern was the high ratio of operations and maintenance (O&M) to salary dollars and inability to convert these dollars (i.e., from O&M to salary), which left many groups with insufficient salary dollars to hire scientific and technical staff. GLSF project proponent and committee key informants were more apt to suggest that the GLAP focus and resources be expanded to include priority areas and activities that extend beyond the designated AOCs.
GLAP IV monitoring and reporting activities are undertaken to meet annual reporting obligations associated with both the GLWQA and the COA. At the level of funded projects, GLSF project reporting is quite complete and the majority of reviewed GLSF files provided clear evidence of intended output achievements. However, limited evidence was provided on the outcomes of remedial actions taken in the AOCs. While federal proponents were supportive of increasing reporting requirements for GLAP projects to enhance performance measurement and accountability, regular reporting of federal science and monitoring projects is not a formal requirement and reporting to the GLAP IV program is ad hoc.
Project proponent key informants indicate a high level of satisfaction with the GLSF program. Program priorities are considered to be clear and appropriate. The application process is regarded as clear and transparent by most funding recipients and selection criteria are viewed to be explicit and well‑communicated. Overall, the GLSF proposal review process is perceived to be logical and GLSF funding decisions are viewed as strategic and fair. Program staff are seen by project proponents to be accessible and supportive. Modifications to the GLSF program (including the shift from the use of MOUs to contribution agreements), implemented by Environment Canada in 2007, have challenged the timely delivery of the program, reduced flexibility and increased uncertainty among partners. Project proponents indicate a preference for streamlining the application and approval process, for multi-year agreements, and for more straightforward funding agreements.
Key informants across the respondent groups had a favourable view of GLAP IV as a program model: the program was generally viewed as a sensible approach to achieve the intended outcomes. No major redundancies or notable gaps were identified within the program.
An important feature of the program, consistently highlighted by key informants across various respondent groups, is the engagement of partners and stakeholders. This occurs in many ways, perhaps most importantly in the formal agreements between Canada and the United States (GLWQA) and Canada and Ontario (COA). These agreements are the foundation for commitments of the parties with respect to the AOCs and are a critical framework in organizing work in the AOCs. Other examples of partner engagement include:
Multiple groups within Environment Canada, including the Canadian Wildlife Service and the NWRI. In addition, the program formally includes eight other federal government departments, which are represented on the GLEC.
RAP committees, which play an important role in driving work in the AOCs. The RAP three-stage reporting process is the basis for updating the GLWQA on progress in the AOCs and the delisting process, as well as for engaging local partners "on the ground." While RAP committees vary in their size and structure, key member representatives can include: federal representatives (e.g., DFO, Environment Canada), other government representatives (Ontario MOE or MNR, municipal/regional, First Nations), Conservation Authorities, community groups and NGOs.
GLSF projects, which require partnerships as a condition of funding approval, particularly (though not exclusively) funding partners. The review of GLSF project files confirmed that all GLSF projects include partners – six on average, with organizations having a local/regional scope of operations being engaged most often (85 per cent of projects), followed by organizations operating at the provincial level (72 per cent) (Table 3.1). Types of partners included: government (federal, provincial, municipal/regional, First Nations) (83 per cent), community-based organization (69 per cent) and the private sector (industry, landowners) and educational institutions (both at 29 per cent). The nature of partner contributions most often included cash and in-kind contributions (e.g., equipment, land, office space or supplies) (97 per cent); followed distantly by assistance with delivery (e.g., planting) or technical advice/assistance (18 per cent each), and participation in an advisory or research capacity (13 and 11 per cent, respectively). "Other" contributions include such things as publicity/promotion and training. Partnerships for federal partner projects were not analyzed in the same way, however, publication listings indicate the involvement of partners as investigators. As well, NWRI project-level documentation for the NWRI shows collaborations with municipalities, the province and other researchers. Contributions include: data sharing; sampling/provision of samples, facilities and field support; and constructing facilities. Leveraged resources are also indicated ($550,000, for two projects out of five in total).
| Scope of partners' operations | Percent of files |
|---|---|
| * Based on a sample of 39 GLSF project files ** The average number of partners per project is 6 |
|
| Local/regional | 85% |
| Provincial | 72% |
| National | 51% |
| International | 5% |
| Sectors represented | |
| Governments | 83% |
| Community-based | 69% |
| Private sector | 29% |
| Educational institutions | 29% |
| Professional associations | 14% |
| Other countries | 3% |
| Other | 14% |
| Partners' contribution | |
| Funding | 97% |
| Service delivery | 18% |
| Technical advice/assistance | 18% |
| Advisory committee | 13% |
| Research | 11% |
| Not described | 5% |
| Other | 13% |
Although federal projects do not have the same requirement for partnering, 16 of 21 federal project files that were reviewed identified partners. These partners typically included Environment Canada or federal OGDs, the Province of Ontario, municipalities, a RAP committee, or university. The nature of partner contributions for federal projects focused most often on collection or sharing of samples or data and information analysis or exchange.
Summary: Engagement of partners represents a critical and beneficial aspect of GLAP IV. Partnerships occur at many levels and in many ways. Formal agreements between Canada and the United States (GLWQA) and Canada and Ontario (COA) represent the foundation for commitments of the jurisdictional parties with respect to the AOCs. Horizontal partnerships within Environment Canada and across the federal government are necessary for GLAP implementation. RAP committees provide a forum for governments (federal, provincial, municipal and First Nations) and key stakeholder groups (conservation authorities, community groups and NGOs) to coordinate their efforts through the delisting process and engaging local partners "on the ground." The funding structure of GLSF projects promotes partner engagement at the local and regional level. Partner contributions include cash and in-kind funding, assistance with delivery, and participation in an advisory or research capacity.
To access GLAP IV funds, federal partner departments, including Environment Canada, each submitted five-year work plans that outlined the projects they proposed to implement using GLAP IV funds and described how each aligned with program priorities. A review team (with representatives from all federal departments that submitted work plans) assessed each of the work plans. In evaluating departmental work plans, priority was given to completing federal actions in revised group 1 AOCs, while revised group 2 AOCs were accorded second priority.[30] As well, the review team did not recommend projects:
The work plans were intended to steer work so that it would be responsive to the needs of the AOCs and be a tool for accountability. In this regard, recipient departments were to report on progress and update their work plans annually, at which time work plan priorities and activities could be adjusted as needed. With the endorsement of the GLEC, the Environment Canada Program Director would sign off on the allocation. While funds for the full five-year work plan period were approved in principle to provide resource stability to organizations, it was also stated that, if necessary, departments would be able to reallocate their total resources based on the reviewed and adjusted/updated work plans. Due to a Department-wide transformation beginning in 2005/06, the GLBEI (of which GLAP IV was a part) was co-led by the RDG‑Ontario and RDG‑Quebec. The intended work planning annual reporting, and review and updating process, never occurred.
Departmental managers and federal partners who were interviewed for the evaluation approved, in general, of the work planning process as a worthwhile exercise to guide activities in AOCs. However, the process as it was intended was abandoned and work plans were not reviewed annually. There was therefore no mechanism available for changing work priorities or reallocating funds on an annual basis (though this latter issue was not universally perceived to be problematic) – the approved projects became "hard‑wired" for the five-year term of the program. As well, the role of the work planning process as a tool for accountability (i.e., to ensure that activities were being conducted by the various partners according to their work plans and to address GLAP IV objectives) did not materialize.
Summary: The five-year work plans required by federal partner departments to access GLAP IV funds and proposed projects were recommended by the review team and approved by the GLEC. Federal key informants consider work planning a positive and worthwhile exercise to guide remedial and monitoring activities in the AOCs. However, the intended work‑planning annual reporting, review and updating process did not occur, which resulted in a loss of flexibility, responsiveness and accountability.
GLAP IV is a horizontal and bilateral initiative, with an AOC-based RAP committee structure (described in subsection 1.1). As a result, contributions to the delivery of federal commitments in AOCs are highly distributed. Commitments of the federal and provincial governments are outlined in the COA, and the GLWQA articulates Canada‑U.S. commitments in the AOCs.
Program documentation with respect to the governance and management of GLAP IV (Management Framework: Great Lakes Action Plan for Areas of Concern (2005‑2010)) does not reflect the current operation of the program in at least one respect. As a result of the departmental transformation that occurred in 2005‑2006, the original structures and authorities for program management were not established in their entirety. The original documentation on the roles and responsibilities for the program identified a Program Management Committee that would have responsibility to support the GLEC, help facilitate the flow of information among the departments, and allow directors/managers the opportunity to brief their senior management on key issues. The Program Management Committee has not been an active committee during the past several years.[31] The communication and coordination function has fallen to the COA Annex Implementation Committee, which has a similar membership. This was not widely noted by key informants, and some respondents perceived a gap in coordination and communication across federal partners.
Despite the horizontal and bilateral nature of the initiative and broad changes in the Department, most federal key informants felt they had a good understanding of their own roles and responsibilities with respect to the program and, at a general level, those of the major partners in the program. GLSF proponents also believed they had a clear understanding of their roles and responsibilities.
However, the roles and responsibilities of federal partners was an aspect of GLAP IV that came under criticism in the CESD's reviews of the program in 2001 and 2008 and the TB MAF exercise with respect to the GLBEI in 2008. The 2008 CESD report noted that "while the department has recently clarified some responsibilities, it has still not clearly specified who is responsible for carrying out all the required remedial actions, who will pay for those efforts, and within what timelines the actions will be taken." Similarly, the Round V TB MAF exercise carried out in 2007‑08 urged the Department to clarify partner roles/contributions and to "review the change to a distributed responsibility model to deliver Great Lakes Water Quality Agreement/Canada-Ontario Agreement commitments to ensure sufficient resource allocation and the ability to meet changing priorities."
While generally aware of GLAP IV roles and responsibilities, a number of key informants echoed some of the CESD and Treasury Board concerns with the "distributed responsibility" model of this horizontal program. For example, there was a feeling among some, particularly those at the working level, that there is a disconnect (or at least the connection is not readily apparent) between the needs of the individual AOCs (as they are articulated in the RAPs) and the remedial as well as science and monitoring projects being carried out by the federal partners. Other key informants noted that, in general, communication and coordination among federal partners is an area that merits further attention from the program. It is difficult to say what type of internal management mechanism or instrument might have most benefit for the program (and key informants, when pressed, did not specify), though there is generally little enthusiasm for additional structures and meetings unless the benefits appear evident.
In response to the CESD report, the program has taken steps in 2009‑2010 to clarify the actions remaining in each AOC, the responsibility of each partner in the AOCs and the proposed timelines. This information, including an update on the status, accomplishments and the responsible partners and timeframes for remaining actions in each AOC, was presented in January 2010 to RAP committees and federal and provincial partners during one of a series of program workshops,[32] and garnered positive reviews (as heard in the evaluation interviews).
Another issue raised by some federal key informants is a grey area surrounding roles and responsibilities with respect to delisting AOCs. The GLWQA states that "The Parties shall cooperate with State and Provincial Governments to classify Areas of Concern by their stage of restoration progressing from the definition of the problems and causes, through the selection of remedial measures, to the implementation of remedial programs, the monitoring of recovery, and, when identified beneficial uses are no longer impaired and the area restored, the removal of its designation as an Area of Concern" and that "The Parties, in cooperation with State and Provincial Governments, shall ensure that the public is consulted in all actions undertaken pursuant to this Annex." In practice, the definition of delisting criteria has proven challenging because of difficulty in determining a target or end point for actions (i.e., defining "when the job is done"). The absence of clear delisting criteria for AOCs came under criticism in the 2008 CESD report as well. Several AOCs are now approaching a state of restoration, but, for some respondents, the authority and process for decision making about delisting are unclear. A small number of respondents noted the potential for a built-in disincentive to delist, due to the resulting termination of GLAP dollars for the AOC and the lack of alternative funding to monitor these areas to ensure that they remain restored. Environment Canada is currently working with the IJC to reach agreement on a consistent set of metrics for recommending restoration of BUIs and delisting of AOCs.
Summary: GLAP IV is a horizontal and bilateral initiative governed by commitments outlined in the GLWQA and COA, with an AOC-based RAP committee structure. Due to the horizontal nature of the initiative and broad changes at Environment Canada, GLAP IV governance and management activities do not reflect those outlined in the management framework developed for the GLAP for AOCs (2005‑2010). Areas identified as a potential source of confusion or concern by federal key informants include: instruments for ensuring that partner contributions occur and are coordinated and consistent with the goals of GLAP IV; and clarity of roles and responsibilities with respect to delisting AOCs.
As indicated, the original allocation for GLAP IV was $40 million over five years. The work planning process identified and recommended funding to federal partners with the intention that amounts would be reviewed and determined in a flexible manner annually, based on evolving needs and identified priorities. In the first year of GLAP IV, allocations were consistent with recommended funding levels (albeit rolled out late in the fiscal year). However, a number of federal partners within Environment Canada noted that the remaining years of GLAP IV saw a significant divergence in the allocation, with some groups such as the Canadian Wildlife Service reportedly receiving substantially less funding (though this could not be confirmed due to the absence of financial tracking information). As well, the annual review (and possible adjustment) of the funding allocation did not occur; funds for subsequent years were allocated to federal partners for the remainder of GLAP IV based on the amount of the first fiscal year allocation.
For those federal partners that received GLAP IV funds, a serious concern was the mix of salary and O&M dollars that were received. DFO and groups within Environment Canada were allocated funds that, in the first year of program implementation, had the flexibility to change O&M for salary dollars and vice versa. A Department-wide initiative subsequently froze these dollars in their respective categories. Because the original allocation was heavily weighted toward O&M, many groups with responsibility for delivering GLAP IV‑funded initiatives had the challenge of insufficient salary dollars to hire the research or technical staff required to carry out the labour-intensive science and monitoring project activities on a long-term basis.
For federal departments and groups within Environment Canada that received their original allocation, the funding amount was generally seen to be "about right," with the caveat that yearly funding amounts for future work in the AOCs would benefit from being increased to account for inflation. In contrast, GLSF project proponents were more apt to argue strongly for an increase in funding to this program component to permit a more aggressive and comprehensive approach to addressing issues in the AOCs. Funding to the GLSF was curtailed after 2005‑06.
A small number of key informants, particularly among the committee members, strongly favoured an increase in GLAP resources. However, this view also included a preference for expanding the terms and conditions of the program beyond AOCs (e.g., expanded AOC boundaries to include watersheds, heavily degraded areas around the lakes that are not AOCs, or the entire nearshore). In other words, while the current level of resourcing for GLAP IV was deemed to be sufficient for the program's narrow focus on AOCs, these respondents would like to see both the resources and programming focus expanded.
A critical issue with respect to resources and financial accountability is the very limited use of financial codes to track GLAP IV program resources. In the June 2006 departmental transformation, Environment Canada determined that priority ecosystem initiatives, including the GLBEI (of which GLAP IV is a part), would be delivered through an unbundling[33] of activities and resources. GLAP IV resources were allocated to Outcome Project Groups[34] within Environment Canada that were responsible for achieving GLAP IV objectives. However, these monies were not coded separately from Great Lakes A-Base resources, nor were they coded separately from resources dedicated to the GLBEI within the Department, thus severely limiting the financial accountability of the program.
Summary: For federal partners and groups within Environment Canada for whom resource allocations were consistent with their original recommended funding levels, allocated GLAP IV funding was viewed to be adequate, with the caveat that future funding should be indexed for inflation. However, some federal groups received significantly less resources than their original approved funding levels, thus negatively affecting their ability to implement planned activities. For federal partners and groups within Environment Canada that received GLAP IV funds, a more pressing concern than the level of funding was the high ratio of O&M to salary dollars, which left many groups with insufficient salary dollars to hire the staff required to undertake science and monitoring project activities. With direct transfer of funds to partners, but no program‑specific coding of expenditures, financial accountability is a significant weakness of the program. GLSF project proponent and committee key informants were more apt to suggest that the GLAP focus and resources be expanded to include priority areas and activities that extend beyond the designated AOCs.
There are reporting obligations associated with the GLWQA and COA to which GLAP IV must respond. The GLWQA requires the parties (Canada and the United States) to report progress on AOCs to the IJC. Comprehensive reporting on AOCs was provided in 2003 to the IJC (resulting in the April 2003 IJC report Status of Restoration Activities in Great Lakes Areas of Concern: A Special Report). An informal briefing on the status of AOCs was also provided to the Canadian Section of the IJC (the Canadian secretary and IJC staff members) in 2009. The Report urged Canada and the United States to: fulfill their commitments to report on "...progress in developing and implementing Remedial Action Plans and in restoring beneficial uses..."; "...document their considerable investment and achievements to date in order to provide the public with a true reflection of their accomplishments."; and "...ensure that monitoring, data support and information management systems are in place..."
Reporting of GLAP IV also occurs annually to update delivery of federal commitments in the COA. Progress in relation to achievement of COA commitments is assessed annually and reported to the COA Management Committee, based on an extensive COA database, where federal and provincial parties input all their projects against each COA result number as per the COA agreement.
At the level of funded projects, GLSF projects follow a reporting template that includes progress reporting (where required) and a final report (or annual report for multi-year projects). Final reports include background/project description, a listing of activities/outputs against intended deliverables, key products (e.g., published papers, research documents), highlights of the project, as well as final financial reporting on project expenditures (GLSF funding and leveraged contributions). The file review confirmed that GLSF project reporting is quite complete: the majority of files contained documentation relevant to the purpose of the project (proposal, technical review), MOU or contribution agreement, as well as a final report and associated products. The file review further confirmed that in the vast majority of files (80 per cent), evidence of intended output achievement was provided to a very or moderately clear degree in the documentation. Three quarters of projects were rated to have a very close or moderate alignment between the project's design and implementation.
Several key informants in various respondent groups pointed out that a challenge in reporting on GLSF-funded projects is measuring the outcomes of the remedial actions taken in the AOCs. While habitat or remediation initiatives may be considered a success in terms of outputs, it is much more difficult to determine the level of success in terms of outcomes (e.g., BUI and AOC restoration). Indeed, the review of files indicated that projects were more likely to have output-related objectives than outcome-related objectives: 38 per cent of files contained output-related objectives only; 15 per cent contained outcome-related objectives only; and 46 per cent contained both types of statements.[35]
As well, the files were far more likely to provide evidence of achievement of intended outputs (80 per cent clear or moderately clear evidence of outputs) than outcomes (54 per cent clear or moderately clear evidence of outcomes). Material on outcomes was more frequently available for projects that had a monitoring component (e.g., water quality or population data) or that were conducted as part of an ongoing multi-year effort. External evaluations of project results were rare.
Reporting requirements for federal science and monitoring projects under GLAP IV are not specified to the same extent as GLSF projects. As a result and as noted previously, the review of federal projects was addressed in a qualitative and descriptive fashion. Reporting on federal projects may include reporting / annual reports to the RAP committees (for whom the science or monitoring was conducted), GLWQA/COA reporting, and contributions to the scientific literature (papers and presentations). For projects conducted by DFO, for example, almost 70 conference presentations were prepared and over 25 reports and publications were prepared for the sample of nine projects funded under GLAP IV.
Regular and structured reporting to the program on project activities has not been made a formal requirement of the funding, and therefore reporting has been sporadic. Federal partners noted that periodic workshop presentations or summaries of activities were requested and were provided to the program on an ad hoc basis.
Federal proponents were generally supportive of increasing reporting requirements for GLAP IV projects, a trend they viewed as being consistent with overall increased attention in government to performance measurement and accountability. While there are some projects, such as basic science, for which performance measures may be difficult to develop for outcomes, respondents were, for the most part, agreeable to enhanced reporting on deliverables outlined in their work plan.
An aspect of reporting that was raised by a small number of key informants was the absence of a mechanism to share the results of science and monitoring activities carried out in the Great Lakes AOCs in order to facilitate technical transfer and share lessons learned. This was a commitment that was set out in the COA Annex 4 pertaining to the AOCs, but was never fulfilled.[36] The development of web-based information-sharing would fall under the Internet Content Renovation Initiative (formerly the One Department, One Website initiative), which moves the Department to a more centralized approval process for publishing of web content. With this, the Ecosystem Sustainability Board is responsible for approving all new web content, and the COA commitment has not been prioritized and approved by the Board. However, GLAP IV has held workshops to facilitate technical transfer. Finally, the lack of inclusion of members of the public or lay audiences in plans for dissemination of activities and results of GLAP IV work in the AOCs was also noted by a small number of key informants as a deficit in program communications.
Summary: GLAP IV monitoring and reporting activities are undertaken to meet annual reporting obligations associated with the GLWQA and COA. At the level of funded projects, GLSF project reporting is quite complete. The vast majority of reviewed GLSF files provided clear evidence of intended output achievements, however, only limited evidence was provided on the outcomes of remedial actions taken in the AOCs. Regular reporting of federal science and monitoring projects is not a formal requirement of the funding program, and reporting to the GLAP IV program over the period reviewed has been ad hoc. Federal proponents were generally supportive of increasing reporting requirements for GLAP projects in order to enhance performance measurement and accountability. There was a weakness identified in information sharing among GLAP IV stakeholders and the general public.
Feedback from GLSF funding recipients indicated a high level of satisfaction with the design and delivery of the program. Almost all project proponent interviewees considered GLSF priorities to be clear and appropriate. GLSF projects are perceived to be aligned with activities outlined in the RAPs developed for the respective AOCs, and are seen by interviewed project proponents as contributing to the overall health of the Great Lakes.
Nearly all project proponents considered the GLSF application process to be clear and transparent. Selection criteria were viewed by most funding recipients to be explicit and well-communicated, with direct links to GLSF priority areas and to delisting criteria indicated in their AOC's RAPs. The GLSF proposal review process was perceived to be logical and GLSF funding decisions are considered to be strategic and fair. According to one project proponent, GLSF is "one of the better-run programs – very focused."
For the majority of funding recipients, the proposal review process was seen as a collaborative and consultative exercise that provides for two-way conversations on the alignment of proposed activities with GLAP IV and RAP priorities and needs. While almost all interviewees were satisfied with the clarity and transparency of the application process, a few project proponents found the process to be overly complex and, depending on the scope and technical nature of the project, noted that it could represent a disincentive for organizations without appropriate in-house technical support, or for smaller projects where the value of the funding sought may not justify the level of effort expended to apply for funding.
The support provided by program staff during the proposal development and review process was seen by nearly all GLSF proponent interviewees as being constructive, with a good level of interaction and support provided. As one project proponent indicated, it is "a very interactive and beneficial process." Most interviewed GLSF recipients indicated that program staff are accessible during the proposal development process, and the feedback and support provided by project officers is helpful. For some project proponents, Environment Canada support enabled them to improve their proposal by enhancing the alignment of the project with the goals and objectives of GLAP IV or by providing technical support for high-profile projects in binational AOCs. A few project proponent interviewees experienced a decline in the level of support provided by program staff, which was attributed to high turnover rates of GLSF staff that occurred during a 2007‑08 program‑restructuring initiative. However, these interviewees also noted recent improvements in the level of support provided by program staff.
While there is a high level of satisfaction in the overall design and delivery of the GLSF program, there is also a high level of consistency among project proponents in the perceived weaknesses of the current design and delivery of the GLSF program. A number of changes to the GLSF approval and funding processes were implemented by Environment Canada in 2007. Notably, the MOU mechanism traditionally used to fund GLSF projects using O&M resources was replaced by contribution agreements to support projects using G&C funds, and the approval process was amended to require senior management and ministerial authorization. The contribution agreement funding mechanism (and associated approval requirements) has compromised the timely start‑up of some time-sensitive on-the-ground projects. Other concerns noted by project proponents included unexpected decreases in GLSF funding, which has led some to consider cancelling or limiting the scope of their projects, and some lack of clarity in the GLSF agreement's numerous clauses.
A majority of funding recipients believe Environment Canada should implement a more streamlined application and approval process to enhance the efficient delivery of the GLSF program by reducing the level of effort expended to access funding (e.g., simplify the application form for non-technical or low-dollar‑value projects, and enable "umbrella" annual proposal submissions by organization) and by shortening the amount of time needed to approve funding proposals and agreements (i.e., simplify the internal review and approval process). A minority of funded recipients indicate a preference for multi‑year agreements that would increase the effectiveness of on-the-ground projects by enabling multi-year project planning, thus ensuring timely start-up of field work by avoiding yearly approval processes.
The issues raised by GLSF project proponents about G&C approval processes are consistent with those identified by the Blue Ribbon Panel on Grants and Contributions in 2006.[37] Since that time, federal departments (including Environment Canada) have been undertaking grants and contributions reform initiatives, which include action plans to, for example, become more client-focused, such as through making the funding process and programs more transparent, reducing the administrative burden on funding recipients, and providing funding to clients on a more timely basis.[38]
Summary: Project proponent key informants indicate a high level of satisfaction with the GLSF program. Program priorities are considered to be clear and appropriate. The application process is regarded as clear and transparent by most funding recipients and selection criteria are viewed to be explicit and well‑communicated. Overall, the GLSF proposal review process is perceived to be logical and GLSF funding decisions are viewed as strategic and fair. Program staff are seen by project proponents to be accessible and supportive. Modifications to the GLSF program, implemented in 2007 by Environment Canada, have challenged the timely delivery of the program, reduced flexibility and increased uncertainty among partners. Project proponents indicate a preference for streamlining the application and approval process, for multi-year agreements, and for more straightforward funding agreements.
This section presents the evaluation findings related to the achievement of the program's intended outcomes, and the identification of any unintended impacts of GLAP IV. Information is also provided on the overall importance of the contributions made by GLAP IV to the restoration and maintenance of AOCs, as well as the identification of external factors that may be affecting, either positively or negatively, the program's activities and outcomes.
In general, key informants had favourable impressions about the performance of the program with respect to the achievement of immediate and intermediate outcomes, a finding that is supported by the program documentation and review of GLSF and federal project files. As indicated, engagement of partners at the local level and across jurisdictions and scientific communities is perceived to be a strength of the program. This is supported by GLAP IV funds allocated to federal partners to facilitate coordination and management of GLAP IV (e.g., with the province and First Nations). Engagement of partners and participation at the local level is facilitated through committees, informal networks and the RAP structure (which also receive GLSF funding support). With respect to remedial actions, addressing pollution (through sediment remediation, support to municipal infrastructure improvements, and reducing non-point sources of pollution) is a significant priority for the program and is allocated almost one half of the GLAP IV dollars. While the success of federal partners' efforts is difficult to discern due to gaps in the federal project files, GLSF projects that supported this objective were found to be well‑documented in terms of outputs and experienced few challenges in implementation. BUI assessment and monitoring work is occurring on many fronts to assess the status of BUIs and the effectiveness of restoration activities, and to define BUI goals/targets. Fewer federal projects were funded for habitat restoration (though this is a particular focus of GLSF). Habitat projects are more easily documented in terms of outputs (e.g., plantings, wetland acreage), but these projects are also more subject to implementation challenges. Overall, program investments advance activity in each AOC to some degree (though some coordination projects target all AOCs generically). Note that while group 1 AOCs (those closest to delisting) were intended to be assigned a higher priority for GLAP IV investments, the number of projects funded in these group 1 AOCs is lower than in the group 2 AOCs.
Achievement of the program's longer-term outcomes is much less evident and the majority of BUIs that were originally identified in AOCs still exist. Of the over 100 BUIs identified, only 20 per cent have been restored. In consideration of the status of the BUIs, the program has identified outstanding priority actions for each AOC and projected timelines for delisting. Only two AOCs have been delisted and one AOC designated an area in recovery, and these occurred prior to GLAP IV. The original program goal of completing federal actions in seven group 1 AOCs has not been achieved. Three AOCs are expected to be delisted or designated as Areas in Recovery within the next 1‑2 years, and a fourth, Wheatley Harbour, has just been delisted (announced on April 16, 2010). With respect to the remaining AOCs, the picture is much more complex and the time frames for delisting are longer-term, with most of these AOCs estimating delisting as occurring between 2015 and 2020. For committee and expert key informants, the overall importance of the contribution made by GLAP IV to the restoration and maintenance of AOCs is viewed as being very significant, if not critical.
For key informants, positive external factors that support program success include: leveraged funding, particularly infrastructure funding; U.S. investments; and community/political engagement (for example, the Great Lakes St. Lawrence Cities Initiative). External factors that interviewees identified as having the potential to negatively affect the success of GLAP IV include: new and changing ecosystem issues, and the economic downturn. Unintended outcomes of GLAP IV cited by key informants tend to be positive, and focus on unexpected interest and engagement of non-targeted groups (e.g., landowners) and the general public as well as on unanticipated opportunities for collaboration and knowledge transfer.
The extent to which GLAP IV has contributed to intended outcomes was explored in key informant interviews, as well as the review of program documents and the file review.[39] In general, key informants who were asked about program-level outcomes had favourable impressions of the performance of the program regarding shorter-term outcomes. Building on the achievements of previous GLAP programs, much work has already been accomplished in identification and characterization of problems in AOCs. The assessment of progress toward achievement of intended shorter-term outcomes identified in the program logic model is based on the perceptions of key informant interviewees, as well as the program documents and files. While descriptions of and allocations to federal projects have been reviewed (i.e., from the original approved work plans), there is limited evidence on the outputs and success of these efforts. Evidence related to the achievement of each of the program's immediate and intermediate outcomes is as follows:
The long‑term outcomes for the GLAP IV are that:
Achievement of the program's longer-term outcomes was an area where the CESD (2008) was critical of the program: "After more than 20 years, only 2 of Canada's original 17 areas of concern have been delisted [Collingwood Harbour and Severn Sound] – the latest in 2003. Priority actions have been completed for one other area, which is now recognized by the government as an ‘area in recovery [Spanish Harbour].' The majority of impairments to beneficial uses that were originally identified in areas of concern still exist today."
Current program data indicate that while many BUIs (21) are assessed as being restored, many more remain impaired (over 80 BUIs across the 15 AOCs) (see Annex I, under separate cover).[40] However, a simple binary restored/impaired measure is not a highly sensitive measure of progress toward restoration and, in some cases, BUIs in various areas may be in the process of being monitored to assess impairment (i.e., to update information on extent/nature of impairment).
The status of BUIs is indicative of overall progress toward delisting of AOCs. Recall that the goal of GLAP IV was to complete federal actions in group 1 AOCs (closest to delisting). Of these seven originally identified AOCs, only three are expected to be delisted or designated an area in recovery by the end of GLAP IV.
Using the work plan Review Committee group 1 priority grouping, three of the seven group 1 priority AOCs were expected to be delisted or designated areas in recovery within the next 1‑2 years: Wheatley Harbour has just been delisted (announced on April 16, 2010); Nipigon Bay is scheduled for delisting in 2011 or 2012, pending infrastructure upgrades; and the St. Lawrence River (Cornwall) AOC has reached the Stage 3 RAP reporting phase and is also scheduled for delisting in 2010 (see Annex I, under separate cover).
With respect to priority group 2 AOCs, the picture is more complex and the timeframes for delisting are longer-term. Several AOCs have now prepared Stage 2 RAP reports or updates (e.g., Detroit River, Niagara River). There are a cluster of AOCs such as Hamilton Harbour and Toronto and Region that require large infrastructure projects to address BUIs in the area. Finally, areas such as St. Mary's River continue with assessment, monitoring and community engagement (Annex I, under separate cover).
While restoration of BUIs and delisting of AOCs has been slower than anticipated, key informants consistently noted the significant challenges of ecological restoration work: heavily degraded areas such as city harbours require expensive infrastructure solutions; ecosystems such as the Great Lakes are complex and dynamic, requiring significant time to diagnose problems, identify solutions and test their effectiveness; and as scientific knowledge evolves, new problems or the need for new solutions are often revealed.
Key informants also highlight the notable successes in several of the AOCs (e.g., Wheatley Harbour and the St. Lawrence River), and the overall importance of the contribution made by GLAP IV to the restoration and maintenance of AOCs. For example, experts and committee member respondents note that GLAP IV funds "focus attention on AOCs," and they generally underscore that federal involvement through GLAP IV funding and scientific expertise is a critical catalyst, stating that "there wouldn't be any movement without it", or that "any holdback now would kill momentum" and that "there would have been [continued] degradation if not for GLAP."
Respondents also noted that despite the critical role played by GLAP IV in the restoration of AOCs, it is not an exclusive one. In monetary terms, GLAP IV funds represent a relatively modest contribution in comparison to larger-scale infrastructure investments. In 2003, the Government of Canada estimated that wastewater and infrastructure improvements across the AOCs would cost approximately $2 billion.
The IJC assists with implementation of the GLWQA, which sets out a series of commitments for Canada and the United States – including a requirement that both countries take remedial action in the heavily degraded AOCs. Using the RAP reporting process, Canada and the United States, as parties to the Agreement, are directed to classify AOCs by their stage of restoration, progressing from
RAPs were developed for all AOCs in the late 1980s when the GLWQA was revised. RAPs are submitted to the IJC for review and comment at three stages. The IJC also adopted an initiative involving status assessments to examine progress toward restoration of beneficial uses in individual AOCs, in an effort to enhance the restoration process. To date, five AOCs remain at the Stage 1 RAP level, though status assessments have been completed for the Detroit River and St. Mary's River AOCs. Stage 2 RAPs have been submitted to the IJC for eight Canadian AOCs and a Stage 3 RAP has been submitted for review for one AOC, Wheatley Harbour (Table 3.2).
| Stage 1 | Stage 2 | Stage 3 |
|---|---|---|
| Source: www.ijc.org/rel/boards/annex2/rap_process.htm | ||
| Jackfish Bay Detroit River St. Mary's River Peninsula Harbour Thunder Bay |
Bay of Quinte Toronto and Region St. Clair River Spanish Harbour St. Lawrence River (Ontario) Niagara River (Ontario) Hamilton Harbour Nipigon Bay |
Wheatley Harbour (now delisted, April 2010) |
Interview respondents across all categories were asked to identify any external factors that might have a positive or negative impact on the restoration and/or maintenance of AOCs. Positive external factors that were seen to have supported program successes include:
External factors that may have had a negative impact on the success of GLAP IV include:
Other challenges or negative external factors identified by respondents include the challenge of having a number of policy renewal activities (i.e., for the GLWQA and COA), which diverts attention from implementation work; and the challenge of restoring and delisting binational sites, due to the jurisdictional complexity and to the generally slower pace of work to date on the U.S. side of the Detroit River.
The GLSF file review also revealed that, on a small number of projects (six), challenges to implementation were identified. These challenges primarily involved project delays, due to issues such as staff turnover, site conditions, and reliance on volunteer effort.
In general, key informants had favourable impressions on the performance of the program with respect to shorter-term outcomes, and this is confirmed by the program documentation and files. Engagement of partners and participation at the local level, a perceived strength of the program, is advanced through various projects to support overall management and coordination, as well as the RAP committee structure. Addressing pollution (through sediment remediation and municipal infrastructure support) is a high priority for the program, and there have been some achievements in this area. BUI assessment and monitoring are ongoing activities that provide scientific support. Habitat restoration is a somewhat lower priority. Priority funding for group 1 AOCs is not currently reflected in the number of projects funded in these AOCs.
Achievement of the program's longer-term outcomes is much less evident, as the majority of impairments to beneficial uses that were originally identified in AOCs still exist. In consideration of the status of the BUIs, the program has identified outstanding priority actions for each AOC and projected timelines for delisting. Three AOCs are expected to be delisted or designated as Areas in Recovery within the next 1‑2 years, and a fourth, Wheatley Harbour, has just been delisted (announced on April 16, 2010). With respect to the remaining AOCs, the picture is much more complex and the time frames for delisting are longer-term, with most of these AOCs estimating delisting as occurring between 2015 and 2020. For committee and expert key informants, the overall importance of the contribution made by GLAP IV to the restoration and maintenance of AOCs is viewed as being very significant, if not critical.
For key informants, positive external factors that support program success include: leveraged funding; U.S. investments; and community/political engagement. Identified external factors that may negatively affect the success of GLAP IV include: new and changing ecosystem issues; and the economic downturn.
Few unintended impacts were identified, either in key‑informant interviews or through the GLSF file review. Five percent of the projects reviewed in the file review identified unintended positive impacts (e.g., school-based partnerships that yielded interest in field courses and the potential to adopt material into the curriculum, as well as opportunities for additional water sampling due to above-average precipitation).
Most unintended impacts identified by interview respondents are positive and involve unexpected public interest in AOC projects or interest among the non-targeted community or groups (e.g., interest and engagement of non-participating landowners in habitat restoration initiatives), and unanticipated opportunities for collaboration among partners or stakeholders.
A second unintended impact noted by a small number of key informants was technical or knowledge transfer to other aquatic ecosystem or other research areas (e.g., tall‑grass seed production and sales arising from habitat work, as well as site habitation data used to identify potential sites for fish nurseries).
Summary: Unintended outcomes of GLAP IV identified by key informants tend to focus on unexpected interest and engagement of non-targeted groups (e.g., landowners) and the general public, and on unanticipated opportunities for collaboration and knowledge transfer.
This section examines the efficiency of GLAP IV activities and delivery, i.e., whether the program is undertaking activities and delivering products in an efficient manner. These evaluation findings include observations on resources leveraged through GLAP IV and administrative costs, as well as perceptions of the efficiency of project implementation and program delivery mechanisms.
The cost-efficiency of GLAP IV is difficult to determine with existing information, due to the unbundling of GLBEI funds and activities and the introduction of a shared, results‑based accountability approach to environmental initiatives. The impact on financial data availability resulting from changes implemented to Environment Canada's resource allocation and accountability structure is that GLAP IV funds are not specifically coded at the program level and the A-base support allocated to GLAP IV is not known.
Financial analysis of the GLSF program component indicates that for every contribution dollar, $0.24 is spent on program administration (salary and O&M). This amount compares favourably to the EcoAction Community Funding Program, which has a cost‑efficiency ratio of $0.39, but unfavourably with the Habitat Stewardship Program ($0.08) or the Invasive Alien Species Partnership Program ($0.13). Like EcoAction, the delivery of the GLSF involves additional staff responsibilities such as participation in RAP committees.
For key informants, there is an impression of efficient program delivery both overall and at the project level, with a number of factors identified that support cost‑efficiency at the program and project levels. Some suggestions to improve efficiency were nonetheless received.
This section presents evidence related to two indicators of cost-efficiency examined in this report – leveraging and administrative efficiency – and discusses challenges associated with each of these analyses.
Leveraging refers to the total value of non-federal contributions to GLAP IV projects. A key challenge related to the calculation of leveraging in the context of GLAP IV concerns the fact that total project costs and partner contributions have not been updated in the electronic program database files since 2006‑2007.
The file review, using a sample of GLSF files, does provide some information on leveraged funding for GLSF projects (Table 3.3). There is a wide range in the value of contributions – from a low of $6,500 to a high of $267,000. Consistent with program guidelines, Environment Canada contribution funding represents approximately one third of the total program cost. A portion of files, 29 per cent, was funded at a higher level, but about half of these are within a few percentage points of the one-third funding ratio and most of the remainder are projects funding RAP implementation.
| Amount of Environment Canada Contribution Agreement Funding | Percent of Files |
|---|---|
| * Based on a sample of 39 GLSF project files | |
| < $50,000 | 36% |
| $50,000‑$100,000 | 38% |
| > $100,000 | 26% |
| Average contribution | $76,678 |
| Total Amount of Funding/Total Cost of Project | |
| < $100,000 | 28% |
| $100,000‑$250,000 | 31% |
| >$250,000 | 42% |
| Average Total Cost of Project | $225,940 |
| Ratio of Funding to Cost | |
| < 33% | 56% |
| 33% | 15% |
| > 33% | 29% |
| Average | 32% |
Project proponents also noted the importance of GLSF resources as a catalyst for leveraged funding. Consistent with the importance of collaboration in Great Lakes restoration and maintenance issues, one key informant confirms that the "federal level has been particularly effective in providing seed money, for example, GLSF provides 1/3 [funding], as well as a stamp of approval, enabling leveraging from other sources." RAP committee key informants identified other ways in which GLAP IV funding served as a catalyst by noting that "GLAP money has been essential to support the Remedial Action Planning which involves many stakeholders" and federal involvement is "important in terms of guidance, experience, coordination, and focusing the efforts."
Unfunded GLSF applicants were asked to indicate whether their project proceeded despite the lack of success in obtaining funding. One of three unfunded applicants interviewed indicates that their project went ahead as planned with several smaller sources of funding. Two of three unfunded applicants interviewed note that their lack of success in obtaining funding has set their projects behind; the projects continue but have been delayed and are proceeding more slowly as they try to secure other funding sources.
Federal projects were far less likely to have leveraged resources from partners. Of the 21 sampled federal project files, only two indicated that the project involved leveraged resources from partners. Leveraged resources therefore represent a small fraction of total project costs (approximately five percent).
Another indicator of program efficiency is derived by examining administrative efficiency (the ratio of operational costs to program dollars). Again, there are limitations in the extent to which this analysis can be conducted, due to departmental changes that occurred during GLAP IV. As indicated, beginning in 2005‑2006 the Department transitioned to a shared, results-based accountability approach to ecosystems initiatives and unbundled GLBEI funds and activities. The aggregate effect of these developments was that GLAP IV expenditures were no longer tagged or coded at the program level by Environment Canada partners. Accordingly, total expenditures and salary, as compared to project or contribution dollars, are not available for GLAP IV overall.
This particular cost-efficiency analysis can therefore be conducted only for the GLSF. However, within this program there have been changes as well, including a merger between the GLSF and the Restoration Program divisions and a reduction in overall contribution dollars, thus making expenditure analysis a challenge.
Based on currently available information (Table 3.4), GLSF project funding over four years totalled $12.66 million. A significant reduction in the annual funding amounts and number of projects funded annually after 2005‑2006 is noted. In total, 223 projects were funded, with the average cost per project being $57,478. With regards to cost-efficiency, for every contribution dollar an average of $0.24 is spent on salaries and O&M for the program. This is lower than the EcoAction Community Funding Program, which has a cost-efficiency ratio of $0.39 but is an intensive community-based delivery model. The GLSF ratio is higher than the Habitat Stewardship Program ($0.08) and the Invasive Alien Species Partnership Program ($0.13). The comparison with EcoAction is likely more appropriate given that GLAP IV staff also undertake significant community-based work (e.g., participation in RAP committees, partnership development).
| 2005 | 2006 | 2007 | 2008** | |
|---|---|---|---|---|
* Final expenditure figures for 2009‑2010 were not available at the time this report was being prepared. ** Estimate only, as the Great Lakes AOC Office and Remediation Program Division were merged in 2008‑2009. |
||||
| O&M | $196,000 | $120,000 | $120,000 | $162,500 |
| Salaries | $653,038 | $842,000 | $587,821 | $398,000* |
| Project Funding Total | $5,423,936 | $2,566,283 | $2,624,950 | $2,047,015 |
| No. of Projects | 95 | 50 | 40 | 38 |
| Avg. Funding/Project | $59,094 | $51,326 | $65,623 | $53,869 |
| Avg. Admin+Salaries as % of Project Funding | 16% | 37% | 27% | 27% |
Program managers and federal partners had favourable opinions about the efficiency of the program. When asked to elaborate on factors supporting program efficiency, interviewees noted the following:
On the other hand, respondents were also asked to discuss factors that undermine program efficiency. The following issues were identified:
Key informants' suggested means of improving cost‑efficiency focused on increasing communication (external and internal) and coordination to increase partner and public engagement in the restoration of AOCs, and to ensure that priority needs in the AOCs (remediation, science, monitoring and engagement) are cohesively linked to work planning. In addition, the ability to hire a stable core of staff was noted as a way to improve program efficiency by enhancing continuity of scientific/technical teams and reducing management time spent on human resources issues.
At the project level, federal and GLSF project proponents were asked about the extent to which their projects' activities and outputs were delivered in the most efficient manner. Respondents generally agreed that their projects were efficient, citing lean operations, collaborations, leveraging of funds from external partners and volunteer efforts. Any impediments to project efficiency were either described as events and constraints beyond the control of the proponents (e.g., staff turnover, weather interfering with fieldwork) or issues pertaining to the GLSF application and funding process. In the case of the latter, a few proponents stressed that constraints, such as the absence of multi‑year funding and the delays in funding approval, present challenges to carrying out projects at peak efficiency.
With respect to GLSF project implementation, the most frequent suggestion to improve cost-efficiency concerned the GLSF application process and approval times. This process was described as "lengthy" and, in the words of one respondent, involves "a great deal of red tape for both government and the proponent" and would require "streamlining the bureaucratic hurdles" in order to improve efficiency. Related to this, a few respondents pointed out that more could be accomplished within the same project (and thus greater efficiency could be achieved) if project approvals were better aligned with the seasonal nature of many projects and/or if agreements were multi-year. Other suggestions included: adopting a highly directive approach to funding GLSF projects to ensure that projects are tightly linked to delisting; and continuing to seek avenues to leverage funding (e.g., private donations, graduate student participation in projects).
The cost-efficiency of GLAP IV is difficult to determine with existing information, due to the departmental transformation that led to the unbundling of GLBEI funds and activities and a loss of program control over activities and funding. Program financial data is limited, as GLAP IV funds were not specifically coded at the program level.
Cost-efficiency analyses were conducted for the GLSF program component only. This analysis indicates that for every contribution dollar, $0.24 is spent on program administration (salary and O&M). This amount compares favourably to the EcoAction Community Funding Program, which has a cost-efficiency ratio of $0.39, but unfavourably with the Habitat Stewardship Program ($0.08) and Invasive Alien Species Partnership Program ($0.13). The GLSF is more comparable to EcoAction given both programs' more intensive involvement with project delivery (i.e., with EcoAction project proponents or with RAP committees for GLSF).
For key informants there is an impression of efficient program delivery both overall and at the project level. Factors identified as supporting program efficiency include: GLSF leveraging, continued directed calls for GLSF proposals, and coordination of science and monitoring activities with provincial and U.S. counterparts. Factors cited as undermining program efficiency refer to the complexity of the initiative (cross‑jurisdictional nature, number of partners and AOCs involved, and complexity of the lakes' ecosystems); the departmental transformation; and the lack of a clear management mechanism for ongoing work planning and reporting. At the project level, lean operations, collaborations, and leveraging of funds and volunteer efforts were viewed as contributing to efficiency, while impediments to efficiency arose from unanticipated events (staff turnover, weather) and inefficiencies in the GLSF application and funding process.
This section explores the extent to which GLAP IV is achieving its intended outcomes in an economical manner. The evaluation findings presented include the perceived cost‑effectiveness of GLAP IV and key informant opinions on alternative, more cost‑effective approaches to program delivery.
There is consensus among key informants that GLAP IV is a good investment of public funds and that results are achieved in an economical way. Design aspects that are seen as contributing to the program's cost-effectiveness include: a focused effort on AOCs; cost‑sharing among jurisdictions and key stakeholders; local involvement and delivery; and continuity of intervention and knowledge acquisition over successive GLAP programs. A few key informants also noted the economic benefits of remediation, as well as the cost of not intervening.
No viable alternative approaches to achieve the same or better outcomes at a lower cost were identified. The prevailing sentiment among key informants is to "stay the course" in that GLAP provides a solid foundation for work in the AOCs. Fundamental changes to the program were discouraged by key informants as they may jeopardize the impact of work completed to date.
The issue of economy concerns whether the achievement of program outcomes occurs in an economical manner and whether there are alternatives to the current program that could achieve the same outcomes at a lower cost. For GLAP IV, it is not possible to quantify total program expenditures due to the absence of financial tracking information. In addition, there is no program point of comparison to assess the relative effectiveness of the program in comparison to other approaches. The analysis is therefore based on key informants' subjective views on the cost-effectiveness of the program.
There was a consensus among key informants across all respondent groups that GLAP IV is a good investment of public funds. While there were some challenges in the implementation of the program (described previously), the majority opinion among key informants was that these challenges did not detract from the overall effectiveness and cost-effectiveness of the program.
Aspects of GLAP IV that were highlighted by key informants as contributing to cost‑effectiveness include the following:
Summary: There is consensus among key informants that GLAP IV is a good investment of public funds and that results are achieved in an economical way. Design aspects that are seen as contributing to the program's cost-effectiveness include: a focused effort on AOCs; cost-sharing among jurisdictions and key stakeholders; local involvement and delivery; and continuity of intervention and knowledge acquisition over successive GLAP programs. A few key informants also noted the economic benefits as well as the cost of not intervening.
Key informants proposed few alternative approaches to the GLAP IV model that would achieve program outcomes at a lower cost. One potential alternative to GLAP IV is the program delivery model used in U.S. AOCs. In the United States, the federal jurisdiction transfers funds to the state level for program implementation and there is a greater reliance on contracted research conducted by universities. This model, while feasible in the Canadian context, was not viewed as desirable by the small number of key informants who discussed the option, including provincial representatives. The transfer of federal funds to other jurisdictions or external organizations was perceived to undermine control over the funding, as well as federal visibility. The continuity of scientific expertise and leadership of the federal government were also viewed as contributing greatly to the success of Canada's efforts in the AOCs. As well, a model that involves contracting-out aspects of work was perceived to increase, not reduce, costs.
Another view, not typically raised as an alternative per se but worth noting here, is the preference expressed by several key informants in various respondent groups to take a broader approach to addressing degradation in the Great Lakes beyond the narrow boundaries of the designated AOCs. These respondents believe there is a need to expand the scope of GLAP priorities (and resources) in order to increase responsiveness to nearshore or lake-wide issues and to promote a more holistic systems-wide approach to addressing remediation of the Great Lakes.
With finite resources, however, key informants were far more likely to suggest a stay the course approach, and generally approved of the GLAP model to restoring AOCs and meeting Canada's commitments under the GLWQA and COA. (Some design and delivery improvements, discussed above, were preferred over potential alternative approaches.) In defence of the current model, key informants noted that there is 20 years of experience with GLAP IV and its predecessors, which has built a foundation of expertise, partnerships and supporting mechanisms (e.g., the GLSF funding program and agreements with other jurisdictions) to move AOCs toward delisting. Moreover, many key informants believe that GLAP investments are now beginning to reach fruition, with several AOCs poised to be delisted or designated an area in recovery.
Summary: No viable options for alternative approaches to achieve the same outcomes at a lower cost were identified. The prevailing sentiment among key informants is to "stay the course" in that GLAP IV provides a solid foundation for work in the AOCs. Fundamental changes to the program are discouraged by key informants as they may jeopardize work completed to date.
[20] See Budget 2010, p.106 (www.budget.gc.ca/2010/pdf/budget-planbudgetaire-eng.pdf).
[21] Environment Canada and U.S. Environmental Protection Agency. 2009. State of the Great Lakes 2009, Highlights. National Wildlife Federation. 2005. Prescription for Great Lakes Ecosystem Protection and Restoration – Avoiding the Tipping Point of Irreversible Changes.
[22] These priority areas include: contaminated sediment assessment; habitat restoration; municipal infrastructure improvements; assessments of ecosystem health; monitoring of water quality and ecosystem health improvements; and management of federal programs and coordination with other partners.
[23] Options included: government, water companies, large companies, citizens, farmers or non‑governmental organizations.
[24] According to the COA Annex, a) Canada and Ontario will co-lead the RAP process in the Toronto and Region, St. Mary's River, St. Clair River and Detroit River AOCs; b) Canada will lead the RAP process in the Thunder Bay, Hamilton Harbour, Port Hope and St. Lawrence River AOCs; and c) Ontario will lead the RAP process in the Nipigon Bay, Jackfish Bay, Peninsula Harbour, Spanish Harbour, Wheatley Harbour, Niagara River and Bay of Quinte AOCs.
[25] Environment Canada. 2009. Status Report on the Implementation of an Ecosystem Approach in Environment Canada.
[26] Environment Canada. 2008. 2007‑2008 Departmental Performance Report. Available at: www.tbs-sct.gc.ca/dpr-rmr/2007-2008/inst/doe/doe00-eng.asp
[27] Government of Canada. "Speech from the Throne". October 16, 2007. http://www.pm.gc.ca/eng/media.asp?id=1859
[28] Environment Canada. 2009. 2008‑2009 Report on Plans and Priorities. Available at: www.tbs-sct.gc.ca/rpp/2008-2009/inst/doe/doe00-eng.asp
[29] http://greatlakesrestoration.us/. Accessed on November 25, 2010.
[30] The review committee assigned first priority to those AOCs with the greatest potential for delisting in the short to medium term (Group 1 AOCs). These Group 1 AOCs include: Thunder Bay, Nipigon Bay, Peninsula Harbour, Wheatley Harbour, St. Lawrence River, Bay of Quinte and St. Clair River. In the COA, Group 1 AOCs are Nipigon Bay, Jackfish Bay, Wheatley Harbour and St. Lawrence River (Cornwall)
[31] With the dissolution of the planning section in the former Great Lakes and Corporate Affairs Branch (concurrent with and as a result of Environment Canada transformation), there was no single focal point to lead the planning function.
[32] These included, in 2004 and 2006, Sharing experiences and a Habitat workshop in 2008.
[33] The department receives a share of its annual budget from temporary (as opposed to core) funding allocations from the Treasury Board in order to deliver programs for various specific purposes. While typically used in the intended manner, the Deputy Minister has the authority to realign this temporary funding to address departmental priorities. Just such a realignment occurred in 2005-06 and this process has come to be known within Environment Canada as "unbundling".
[34] OPGs were the basis of the new management structure in the department to promote integrated management and decision-making in the context of a clearer view of departmental results and strategic direction.
[35] Examples of outcome-related objectives include: rehabilitation of aquatic and riparian habitat resulting in re‑establishment of fish and wildlife; and monitoring and reducing rural non-point source pollution.
[36] Goal 2 in Annex 4 is to "Continue to improve the discovery and sharing of data, information and trends in the Great Lakes Basin Ecosystem" with the expected result of "Increased sharing of data and information among governments, organizations and Basin residents", including through web-based means.
[37] Treasury Board of Canada Secretariat. 2006. From Red Tape to Clear Results: The Report of the Independent Blue Ribbon Panel on Grant and Contribution Programs. Ottawa.
[38] Environment Canada. May 2009. Grants and Contributions Reform – Presentation to Executive Management Committee.
[39] Interviews with federal partners and RAP committee members focused on success relating to specific program outcomes, while interviews with GLSF proponents, experts and committee members focused on success in more general terms.
[40] In 2005, there were 98 BUIs.
[41] In fact, Canada has renewed its commitment to GLAP in Budget 2010 for $8 million per year (http://www.budget.gc.ca/2010/pdf/budget-planbudgetaire-eng.pdf p106)