This page has been archived on the Web

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Skip booklet index and go to page content

Evaluation of Freshwater Programs under the Action Plan for Clean Water

Findings

This chapter presents the findings of the evaluation of Freshwater Programs, presented by evaluation issue (relevance and performance) and by the related evaluation questions. The findings at the overall issue level are presented first, followed by the findings for each evaluation question.

A rating is also provided for each evaluation question based on the evaluation findings. The rating symbols and their significance are outlined below in Table 3.1. A summary of ratings for the evaluation issues and questions is presented in Appendix C (under separate cover).

Table 3.1. Rating Symbols and Significance
SymbolSignificance
AchievedThe intended outcomes or goals have been achieved or met
Progress Made; Attention NeededConsiderable progress has been made to meet the intended outcomes or goals, but attention is still needed
Little Progress; Priority for AttentionLittle progress has been made to meet the intended outcomes or goals and attention is needed on a priority basis
N/AA rating is not applicable
~Outcomes achievement ratings are based solely on subjective evidence

3.1 Relevance

This section on the issue of relevance presents the evaluation findings on the continued need for the Freshwater Programs, the alignment of the Programs with federal and departmental priorities, and the extent to which the Freshwater Programs are consistent with federal roles and responsibilities.

Overall Findings

The evaluation results indicate that there is a continued need for restoration and remediation in the Lake Winnipeg, Lake Simcoe and Great Lakes freshwater systems. Each of the three basins targeted by the Freshwater Programs is experiencing continuing pressures and degraded aquatic ecosystem health. The views of key informants and public opinion data are also supportive of efforts to address the ecological health of freshwater systems. There were no significant concerns about overlap and duplication between the Freshwater Programs and other funding sources.

The Freshwater Programs are consistent with current federal and departmental priorities. The Freshwater Programs are integral to the Government of Canada’s Action Plan for Clean Water. The Freshwater Programs align well with Environment Canada’s strategic objectives to ensure that Canada’s natural environment is conserved and restored for present and future generations.

Documentary and key informant evidence indicates that the Freshwater Programs are consistent with federal and departmental roles and responsibilities. Legislated mandates for both the federal government and Environment Canada include responsibility for freshwater quality, although these responsibilities are often shared with other jurisdictions and/or departments. The federal mandate is strongest for the Great Lakes AOCs, given the federal government’s clear jurisdictional authority over these transboundary waters and its obligations under international and provincial agreements. There is also a strong mandate for continued federal programming within the Lake Winnipeg Basin given the transboundary nature of the watershed and the recent Canada–Manitoba MOU Respecting Lake Winnipeg and the Lake Winnipeg Basin. The evidence for a continued federal presence in the Lake Simcoe Basin is weaker.

3.1.1 Continued Need

Evaluation Issue: Relevance

1. Is there a continued need for the Freshwater Programs?

Evaluation Issue: Relevance
Indicator(s)MethodsRating
Demonstration of societal / environmental need
  • Document review
  • Literature review
Achieved
Presence / absence of other programs that complement or duplicate the objectives of the Freshwater Programs
  • Document review
  • Key informant interviews
Achieved
Gaps would exist in addressing societal / environmental needs in the absence of the Freshwater Programs
  • Document review
  • Key informant interviews
Achieved
Reach and activities are connected to societal / environmental needs
  • Document review
  • Key informant interviews
Achieved
Views on connection of program objectives with societal / environmental needs
  • Key informant interviews
Achieved

Connection to Environmental / Societal Needs

The evaluation evidence supports the continued need for programming to address the health of the aquatic ecosystems targeted by the Freshwater Programs, which are degraded and under pressure due to factors such as urban development, and agricultural and industrial activities. The following sections present this evidence for each of the freshwater systems.

Great Lakes Basin

Documentary evidence indicates that the Great Lakes Basin is ecologically important and its environmental quality is essential for millions of Canadians. It is home to about one third of Canada’s population, contains 8 of Canada’s 20 largest cities and provides drinking water for over 8 million Canadians. The basin plays a vital role in the physical, social and economic life of Canada, and supports almost 40% of Canada’s gross national product, 25% of its agricultural production and more than 50% of its manufacturing activity. It also supports rich biological diversity and significant fisheries. Because they represent approximately 20% of the world’s surface fresh water, the Great Lakes also have international significance.12

Population growth and economic development in the Great Lakes Basin during the 20th century have degraded the environmental quality of the aquatic ecosystem and are threatening the economy and quality of life of the people who depend on the Lakes. For over 20 years, Canada and the United States have been committed to restoring beneficial uses in AOCs in the Great Lakes. The State of the Great Lakes 2009 report provides an assessment of the Great Lakes Basin ecosystem based on a number of ecosystem health indicators.13 The status of the Great Lakes ecosystem was assessed as mixed; some conditions are improving and some are deteriorating. While there have been efforts to address degradation and contamination in AOCs through the implementation of remedial action plans, the Great Lakes continue to experience stress from a combination of sources that include toxic contaminants, invasive species, emerging chemicals and eutrophication. Degradation of the nearshore zone was recently identified by the IJC as a particular threat in the Great Lakes.14

Lake Simcoe

Lake Simcoe is the fourth-largest lake in the province of Ontario. Located north of Toronto, Lake Simcoe is a source of drinking water for 8 municipalities and assimilates waste from 15 sewage treatment facilities. Lake Simcoe plays a vital role in the region’s economic prosperity and well-being.15 Recreational activities are estimated to inject more than $200 million annually into the local community. Overall, 47% of the Lake Simcoe watershed’s land area (approx. 2,800 km2) is agricultural. Agriculture contributes at least $300 million per year to the provincial economy.16 Lake Simcoe also supports a wide variety of aquatic animals, including coldwater fish, warmwater fish, invertebrates, amphibians and reptiles, and benthos and plankton. Sixty-five species within the watershed ecosystem are rare and 33 of these species are at risk.

Due to rapidly increasing population growth, urban development and agricultural intensification, ecosystem health in Lake Simcoe has been steadily declining for many years. Excessive amounts of phosphorus pollution from runoff from urban, rural and agricultural uses in the watershed are contributing to excessive algae growth. As a result, the lake is being robbed of oxygen, which is negatively impacting coldwater fish, wildlife and overall water quality.17

In 2008, the Ontario government passed the Lake Simcoe Protection Act (LSPA) to address water quality concerns and other threats to the lake. The LSPA represents unique watershed protection legislation in Canada and may be a model for the application of the ecosystem approach. Under the Act, the province released

  • a comprehensive protection plan (June 2009) with legally binding policies that address cumulative impacts of pollution that establishes a phosphorus loading target of 44 tonnes per year (a 40% reduction);
  • a phosphorus reduction strategy (July 2010) that establishes sector specific targets, proportionally based on current loading contributions; and
  • a water quality trading feasibility study (July 2010), which concluded that water quality trading18 is feasible within the watershed and could play a meaningful role in reducing phosphorus loads.

In September 2010, Ontario’s Environmental Commissioner commended the ambitious targets and the focus on cumulative impacts, and recommended that this pollution-loading approach be expanded to the Great Lakes.

Lake Winnipeg Basin

The Lake Winnipeg Basin extends into four provinces and four states,19 and most issues affecting the basin have an interjurisdictional aspect. Documentary evidence indicates that the water quality in the Lake Winnipeg Basin has experienced significant degradation. The Lake Winnipeg Research Consortium has warned that Lake Winnipeg is reaching a state of deterioration that may affect ecosystem sustainability.20Similarly, a recent study on sudden ecosystem state change concluded that damage from nutrient pollution to Lake Winnipeg may be reaching a stage where it becomes irreversible.21

In June 2011, the Government of Manitoba tabled Bill 46: the Save Lake Winnipeg Act. In the preamble, the government noted that, despite significant progress, Lake Winnipeg and its watershed continue to receive excessive amounts of phosphorus and nitrogen that result in algae blooms of increasing intensity and frequency, and that recent studies demonstrate that without significant reductions of nutrients and phosphorus, Lake Winnipeg’s ecological health will continue to decline, making restoration efforts increasingly more difficult or impossible.

View of key informants and public opinion

While the science base confirms that each of the three basins targeted by the Freshwater Programs are experiencing continuing pressures and degradation of their aquatic ecosystem health, the perceived need for continued federal support is not consistent across the three ecosystems among EC personnel, federal representatives and experts. The need for federal support was perceived to be greatest for the Great Lakes AOCs, given the immense social and economic value of the Great Lakes for all Canadians, the diverse and complex environmental issues impacting the aquatic ecosystem, and the strong established scientific base that supports the need for restoration.

Support was strong among federal representatives and project proponents for continued federal support in the Lake Winnipeg Basin based on the basin’s transboundary waters, the geographical significance and environmental importance of the watershed, significant social and economic value, and the extreme degradation of Lake Winnipeg. Key informants also noted the limited availability of alternative funding sources and the need to build local science and community capacity in this basin area.

Public opinion is consistent with scientific and key informant views on the importance of Canada’s fresh water resources and the continued need for federal support in their restoration and maintenance. A 2009 Nanos poll found that 6 in 10 Canadians (62%) consider fresh water our most important natural resource, compared to 2 out of 10 (22%) who consider oil and gas to be the most important. Also, 4 in 10 Canadians (40%) indicate that water pollution from industry, agriculture and/or households is their “greatest concern” regarding freshwater in Canada.

Canadians’ confidence in the supply and safety of our freshwater resources has declined significantly over recent years. The 2009 Water Attitudes Study found that, while nearly all Canadians believe it is important to conserve freshwater on an ongoing basis (95%), confidence that Canada has enough freshwater for the long term has declined by 11 points, from 81% in 2008 to 70% in 2009. Similarly, Canadians’ confidence in the safety of Canada’s freshwater supply has also declined, from 81% in 2008 to 72% in 2009 (9 points). Seven in 10 Canadians (70%) consider “government” to be the “most responsible” for ensuring the availability of clean water for their communities.22

Potential for Duplication and Overlap

Documentation, key informant interviews and the review of project files point to the availability of a number of other funding sources that support remediation, science and/or monitoring in the freshwater ecosystems targeted by the Freshwater Programs. Alternative funding sources include

  • Environment Canada A-base funds (salary dollars for permanent staff involved in science and monitoring in the AOCs and elsewhere);
  • other Environment Canada funding programs (e.g., Habitat Stewardship Program, EcoAction, Environmental Damages Fund) that may fund projects in the designated aquatic ecosystems (but not co-fund Freshwater-funded projects);
  • other federal departments such as DFO, AAFC and PC;
  • provincial freshwater programs such as the Lake Simcoe Protection Plan and the Lake Winnipeg Action Plan;
  • other provincial ministries such as those whose mandates concern the environment, natural resources, or agriculture;
  • regional conservation authorities/districts;
  • municipal and township funding;
  • foundations (e.g., the Trillium Foundation), associations (e.g., Ontario Federation of Anglers and Hunters), non-governmental organizations (Ducks Unlimited), and universities;
  • Infrastructure Canada funding for infrastructure projects, such as wastewater treatment improvements; and
  • industry funding.

While most interviewees are aware of alternative funding sources that support the restoration and maintenance of environmentally degraded areas in the three aquatic ecosystems targeted by the Freshwaters Programs, few expressed concerns about duplication or overlap between the Freshwater Programs and other funding sources.

For contribution programs, the risk of duplication between funded projects and other funding sources is mitigated by the cross-jurisdictional composition of the technical review committees set up to review project proposals for the LSCUF and LWBSF components, and to identify potential areas of funding overlap or areas where efficiencies among projects can be achieved. Contribution funding for restoration and research projects is seen as being complementary rather than overlapping by the majority of interviewees.

Most key informants agree that without federal funding for Freshwater Programs, programming and achievement of intended outcomes in the targeted aquatic ecosystems would be impacted. For the Great Lakes, the absence of federal funding would have an impact on Canada’s ability to complete sediment remediation projects to meet Canada’s international commitments under the GLWQA. For Lake Winnipeg, the absence of Freshwater Programs funding would weaken the province’s ability to implement the required governance, science and remediation frameworks, thus delaying and reducing the scope of the restoration and maintenance of the basin. For Lake Simcoe, the absence of Freshwater Program funding would reduce on-the-ground remediation work, science and monitoring work.

Most interviewed project proponents (funded and unfunded) indicate that they applied for funding for their project through other funding sources in addition to the Freshwater Programs. Approximately half of these interviewees were successful in their application. All project proponents indicated that their project would have been negatively impacted if they had not received LSCUF or LWBSF funding, with the majority indicating that the project activities would have been reduced and a few indicating that the project would not have gone ahead. Case studies of projects implemented by the Freshwater Programs also confirm the importance of federal support to secure other sources of funding and as a catalyst for building project support among stakeholders.

Summary

Documentation and key informant opinion indicate that there is an ongoing need for support for remediation and maintenance of targeted aquatic ecosystems. Public opinion data also suggest broad support for efforts to address water quality in freshwater systems. All three basins targeted by the Freshwater Programs are experiencing continuing pressures and degraded aquatic ecosystem health. Key informants also noted the importance of federal jurisdictional authority due to the transboundary nature of the Great Lakes and the Lake Winnipeg watershed. There were no significant concerns about overlap and duplication between Freshwater Programs and other funding sources. Alternative funding sources for restoration are best characterized as complementary. The risk of duplication or overlap is mitigated through formal and informal coordination mechanisms (e.g., the interdepartmental technical committees and public advisory committees established for the stewardship funding programs, Canada–Manitoba MOURespecting Lake Winnipeg and the Lake Winnipeg Basin). The funding available for stewardship projects has augmented other funding available in this area and, for proponents, provides credibility in seeking leveraged funding and building relationships with other stakeholders.

3.1.2 Alignment with Federal Priorities and Departmental Strategic Outcomes

Evaluation Issue: Relevance

2. Are the Freshwater Programs aligned to federal government priorities?

Relevance
Indicator(s)MethodsRating
Freshwater Programs’ objectives correspond to recent/current federal government priorities
  • Document review
Achieved
Freshwater Programs’ objectives are aligned to current Environment Canada strategic outcomes
  • Document review
Achieved
Views on the alignment of program objectives to recent/current federal government and Environment Canada priorities
  • Key informant interviews
Achieved
Federal priorities

Documentary evidence indicates that Environment Canada’s Freshwater Programs are aligned to support federal government priorities established under the 2007 Speech from the Throne. The 2007 Throne Speech commits the Government of Canada to implementing a new water strategy to help clean up major lakes and oceans. Budget 2007 created the Action Plan for Clean Water, which allocated $105.3 million in funds to improve the quality of water in Canada’s rivers, lakes and oceans, including funding for the Freshwater Programs. For the Great Lakes, ongoing federal support is further evident in Canada’s current negotiations for the next GLWQA and COA, and renewal of the Great Lakes Action Plan.

Interviewees also noted the complementarity of Freshwater Programs with federal strength in science and monitoring and with other federal work on addressing toxic substances and supporting sustainable farming practices.

Departmental priorities

In its 1987 Federal Water Policy, Environment Canada identified two goals: to protect and enhance the quality of Canada’s water resources; and to promote the wise and efficient management and use of water. To achieve these goals, the policy identified strategies for science leadership, integrated planning, legislation and public awareness.

Environment Canada’s work plays a key role in providing the science leadership required by all Canadian jurisdictions to inform the sustainable management of Canada’s aquatic resources. In its 2011–2012 Report on Plans and Priorities, providing the science support and delivering on the federal Action Plan for Clean Water were identified as key priorities for the Department.

The Freshwater Programs fall under the Department’s strategic outcome “Canada’s natural environment is conserved and restored for present and future generations.” In the 2011–2012 Departmental Program Activity Architect (PAA), the Freshwater Programs are a sub-sub-activity under Sustainable Ecosystems program activity (PAA 1.3), Ecosystems Initiatives sub-activity (1.3.4)).23

The Priority Ecosystem Initiative Management Framework (PEIMF), developed in 2007 and implemented in 2008 (after approval of the Freshwater Programs), is a tool that Environment Canada now uses to implement ecosystem-based management through the systematic selection, and design and delivery of integrated and partneredpriority ecosystem initiatives. Analysis under this framework identifies the Great Lakes as a priority ecosystem,24 and the Lake Winnipeg Basin as a priority hotspot (Lake Simcoe was not assigned a designation).

Summary

The evaluation found the Freshwater Programs to be consistent with current federal and departmental priorities. The Government of Canada continues to move forward on its Action Plan for Clean Water. The Freshwater Programs align well with Environment Canada’s strategic goal to ensure that Canada’s natural environment is conserved and restored for present and future generations.

3.1.3 Consistency with Federal Roles and Responsibilities

Evaluation Issue: Relevance
3. Are the Freshwater Programs consistent with federal roles and responsibilities?
Indicator(s)MethodsRating
Freshwater Programs mandate aligned with federal government jurisdiction, mandate and roles and responsibilities
  • Document review
Progress made, attention needed
Views on the appropriateness of federal involvement
  • Key informant interviews
Progress made, attention needed

The federal and provincial governments share responsibility for freshwater management. The provinces have authority to legislate with respect to various aspects of water supply and use, pollution control, irrigation and recreation within their borders. The federal government has jurisdiction over aspects of fresh water regulation, including fisheries, navigation, boundary and transboundary waters shared with the United States (as in the case of the Great Lakes and Lake Winnipeg) and federal lands25. Key federal legislation, arrangements and initiatives that support freshwater management include the following:

  • Department of the Environment Act;
  • Fisheries Act
  • Canadian Environment Protection Act, 1999
  • Canadian Environmental Assessment Act
  • International Boundary Waters Treaty Act
  • International River Improvements Act
  • Navigable Waters Protection Act
  • Canada Shipping Act
  • Canada Water Act;
  • Prairie Provinces Water Board Water Quality Agreement
  • Canada–Ontario Agreement Respecting the Great Lakes Basin
  • Great Lakes Water Quality Agreement
  • federal Action Plan on Clean Water

Environment Canada is the federal lead department on a wide range of environmental issues with respect to freshwater systems and addresses these issues through research, policy development, service delivery to Canadians, regulations and enforcement of environmental laws, and strategic partnerships. The Department of the Environment Act assigns national leadership to the Minister of the Environment in matters relating to Freshwater Programs, among others, such as

  • water management;
  • the enforcement of any rules or regulations made by the International Joint Commission relating to boundary waters;
  • the preservation and enhancement of the quality of the natural environment, including water;
  • renewable resources, including migratory birds and other non-domestic flora and fauna; and
  • the coordination of policies and programs of the Government of Canada respecting the preservation and enhancement of the quality of the natural environment.

Nearly all interviewees agree that it is appropriate for the federal government to play a role in the restoration and maintenance of healthy priority aquatic ecosystems. The federal level was identified as playing a leadership role in national water governance and has an established science capacity and a social mandate to protect Canada’s freshwater resources.

The federal government is seen as having a strong jurisdictional authority in the transboundary waters of the Great Lakes and Lake Winnipeg basins. In the Great Lakes, Canada also has obligations under international and federal-provincial agreements. The need for federal support for remediation of Lake Simcoe was less evident for EC management and staff. Given that the watershed boundaries are contained within the province, there is a strong established watershed governance and science base, and environmental issues are primarily local in nature.

Summary

Water governance in Canada is complex, with both federal and provincial jurisdictions having relevant authorities. At the federal level, there are a number of legislative authorities that support the federal jurisdiction and Environment Canada leadership in matters relating to water quality (notably, the Department of the Environment Act, the Canada Water Actand CEPA 1999). The transboundary nature of the Great Lakes waters and Canada’s commitments under the GLWQA provide a strong mandate for federal jurisdiction in sediment remediation in the AOCs. Similarly, the Lake Winnipeg Basin extends across four provinces and four states, arguing for a federal role in its governance. Lake Simcoe’s watershed is provincially bounded, however, and thus the federal mandate is weaker in this lake.

3.2 Program Performance: Achievement of Intended Outcomes

Overall Findings

Evidence from all lines of inquiry indicates progress toward the achievement of immediate outcomes for the LWBSF and LSCUF, although progress has not been consistent across all outcome areas. The evaluation evidence also points to progress toward the achievement of intermediate outcomes, although the progress achieved to date is preliminary in nature and there are challenges in measuring and attributing changes in phosphorous levels in the lakes.

With respect to immediate outcomes, funding available through the LWBSF and the LSCUF has been fully committed to projects led by basin groups targeted for funding (since the local conservation authorities have led a substantial portion of projects, particularly in Lake Simcoe). Stakeholder support is evident through the wide array of project collaborations that were established. All LWBSF and LSCUF projects leveraged funding from other stakeholders and most LWBSF and LSCUF projects leveraged both monetary and in-kind contributions. Collaborators who provided financial support to projects were drawn from a broad spectrum of targeted stakeholder groups, although participation from industry was limited. Governance (Lake Winnipeg) and science, research and monitoring (Lake Winnipeg and Lake Simcoe) were addressed under the Freshwater Programs. The key governance achievement for the LWBI was the Canada–Manitoba MOU Respecting Lake Winnipeg and the Lake Winnipeg Basin. Science objectives are being addressed in Lake Winnipeg through a science plan with six objectives and clear deliverables to enhance dissemination (through an information portal) and to support decision making by establishing nutrient objectives and performance indicators for the lake. Research and monitoring projects in Lake Simcoe were designed to address gaps in knowledge, as the science foundation pertaining to this lake is strong overall.

With respect to intermediate outcomes, the evaluation evidence indicates that funded stewardship projects in the Lake Simcoe and Lake Winnipeg basins have contributed to the reduction and/or diversion of phosphorus entering these watersheds. Although the impact of LSCUF and LWBSF stewardship projects is relatively small in relation to the overall volume of phosphorus entering the watersheds, in many instances these contributions are expected to be cumulative over time. Use of BMPs in Lake Winnipeg and Lake Simcoe could not be assessed directly due to limited performance tracking of this activity specifically. However, some initial work was conducted in this area with project proponents and reasonable progress was perceived to have been made.

LWBSF public education and communications projects have been funded to contribute to the reduction of phosphorus loading by increasing awareness among the general population of the need to modify behaviours (e.g., the Lake Friendly Campaign). The campaign has been extended geographically and in its scope of activities, which is an indicator of success, though awareness levels and impacts on behaviours have not been measured directly.

Progress in the implementation of sediment remediation projects in the Great Lakes is mixed. There has been significant progress in the management and removal of sediment in three of the eight AOCs targeted by the GLSRP. However, for sediment remediation projects in the remaining AOCs, the original timelines and cost estimates are no longer realistic due to a number of unanticipated challenges, such as the uniqueness of site conditions, the need for additional time to fully characterize the sites, and difficulties in securing funding from community stakeholders. Addressing BUIs and delisting are long-term objectives in these AOCs, with sediment remediation being only one of many actions required for delisting. The GLSRPcontributes to addressing Canada’s international commitments under the GLWQA and gaps noted by the IJC in funding for sediment remediation.

There are many factors external to the GLSRP that influence the achievement of intended outcomes, including the dynamic and complex nature of the ecosystem itself, multi-jurisdictional/ownership relationships that must be navigated (especially in the context of the AOCs), and economic constraints of other stakeholders to share the cost burden of restoration.

3.2.1 Immediate and Intermediate Outcomes

Evaluation Issue: Performance
4. To what extent have intended outcomes been achieved as a result of the Freshwater Programs?
Indicator(s)MethodsRating
Evidence of / views on intended output and outcome achievement
  • Document review
  • Program performance and financial data analysis
  • Key informant interviews
  • Case studies
Progress made, attention needed
Evidence of factors outside the program that have influenced the achievement of intended outcomes
  • Document review
  • Program performance and financial data analysis
  • Key informant interviews
  • Case studies
Progress made, attention needed
Views on the extent to which intended outcomes have been achieved as a result of the program
  • Key informant interviews
Progress made, attention needed

This section presents the evaluation findings related to the achievement of the Freshwater Programs intended outcomes. For the GLSRP, program outcomes are assessed separately from the intended outcomes that are common to both Lake Simcoe and Lake Winnipeg. Finally, ultimate outcomes for the Freshwater Programs overall are examined. The extent to which the Freshwater Programs have contributed to intended outcomes was explored in key informant interviews, as well as the review of program documents and the file review and case studies.

Lake Simcoe and Lake Winnipeg – Immediate Outcomes

  • Lake Winnipeg and Lake Simcoe basins groups targeted for funding are aware of and access the Stewardship Fund and Clean-Up Fund respectively

The targeted groups for funding (according to the program terms and conditions) are non-profit organizations; provincial and municipal government departments and organizations; regional conservation authorities and districts; for-profit organizations and individuals such as landowners and farmers; post-secondary institutions; and Aboriginal bands.

Based on the sample of funded projects drawn for the file review, project proponents are representative of the program target group for funding, with regional conservation authorities/districts undertaking a majority of projects (58%). Project proponents also include, to a lesser extent, representatives of other levels of government (11%) and educational institutions (9%).

Table 3.2: Characteristics of Funded Groups

Table 3.2 presents the characteristics of groups funded under the three Freshwater Programs.
TotalFiles (%)
Totaln=55
Project Proponent 
Conservation authority58
Provincial/municipal government11
Educational institution9
Environment NGO4
Business/landowner4
Community group2
Agricultural association2
Other11

Awareness of the contribution funding programs among the program funding target was perceived to be satisfactory by LWBI and LSCUF key informants. In Lake Winnipeg, a communications plan for the LWBI was designed in part to “inform the various community groups, organizations, levels of government and other eligible applicants of the availability of EC’s Lake Winnipeg Stewardship Fund.” In both Lake Winnipeg and Lake Simcoe, communications vehicles such as an updated departmental Internet presence, attendance at special events/ conferences, direct mail, print media, and news releases and backgrounders were used to increase visibility of the LWBI and LSCUF activities and of stewardship funding opportunities among target audiences.

  • Funding is leveraged through project collaborators

The terms and conditions for contribution funding for the LWBSF and LSCUF are flexible in terms of their leveraging requirement. The Programs’ terms and conditions indicate that the federal government will seek a cost-sharing ratio of from one third to two thirds from the provincial governments and other groups that propose projects to access contribution funding. Based on a review of a sample of projects funded by contribution agreements under the Freshwater Programs, all projects leveraged funding--either a monetary (87%) or in-kind contribution (98%). Four in 10 projects also received support from other collaborators in the implementation or delivery of their project (41%). In addition, many projects set up technical or science advisory committees to review and guide activities undertaken. The average federal contribution was 41% of the total project cost. A summary of project funding is provided below.

Note that internal tracking indicates that a somewhat higher proportion of project costs tended to be leveraged in the early years of the Programs: For instance, according to the Progress Report on the Lake Winnipeg Basin Initiative 2008/09 – 2009/10, for every dollar approved through the LWBSF, approximately $3.92 in cash and in-kind contributions were generated. Similarly, according to the Progress Report on the Lake Simcoe Clean-Up Fund 2008–2009, for every dollar of the Fund, four dollars of in-kind contributions were provided by project collaborators to complete the projects, as well as hundreds of people contributing thousands of hours of volunteer time. The difference in the amount of leveraged funding between internal program analyses and the file review data indicate that leveraging may have decreased over the Program life cycle, or may reflect some vagaries in the reporting or calculation of leveraged contributions between the two analyses.

Table 3.3: ECFunding and Leveraged Funding

Table 3.3: ECFunding and Leveraged Funding
 Files (%)
Totaln=55
Contribution from other collaborators 
Monetary contribution87
In-kind98
Project implementation/delivery41
Advisory committee9
Research6
Other (marketing and technical support)2
Amount of EC contribution agreement funding 
< $25K31
$25K–$49K29
$50–$99K16
$100K+24
Average contribution$74,766
Total amount of funding / total cost of project
< $50K18
$50K–$149K46
$150K–$249K7
$250K+22
Unknown7
Average total cost of project$211,048
Ratio of funding to cost 
< 33%31
34%–49%29
50%–66%31
67%+2
Unknown7
Average41%
  • Funded priority projects for point and non-point source nutrient reduction and habitat restoration in the Lake Winnipeg and Lake Simcoe basins have broad stakeholder support

While a systematic canvassing of stakeholder opinion (e.g., through a survey) was beyond the scope of the evaluation, stakeholder support for funded priority projects can be demonstrated through collaborations established at the project level (signalling credibility and endorsement of the project). Based on the sample of contribution agreements selected for the file review, funded projects established an average of 5 collaborations with stakeholders. Collaborators almost always included organizations whose scope of operations was local/regional (94% of projects) and more than half included provincial organizations (56%). Collaborators were drawn from a variety of sectors, including provincial/municipal governments (89% of projects), community-based/non-profit organizations (51%), regionally based conservation organizations (36%), the private sector (e.g., landowners) (24%), and educational institutions (17%).

Table 3.4: Characteristics of Freshwater Programs Contribution Agreements: Collaborations

Table 3.4: Characteristics of Freshwater Programs Contribution Agreements: Collaborations
 Files (%)
Totaln=55
Number of partners identified 
Average5
Scope of collaborators' operations (multiple mentions possible)
Local / regional94
Provincial56
National15
International6
Sectors represented(multiple mentions possible)
Federal government7
Provincial / municipal governments89
Community-based / non-profit51
Private sector24
Educational institutions17
Professional associations9
Landowner11
Other (foundations, volunteers)11
Conservation authority36

In addition to support at the project level, stakeholder support is also evident at the program level. The LWBI and LSCUF programs have established a number of mechanisms and hosted opportunities by which stakeholders might participate in communication and dialogue: executive/implementation committees; technical committees; public advisory committees; and meetings/workshops/forums.

  • Watershed governance mechanisms are in place to facilitate coordination, integrated decision-making and collaboration to improve the water quality and ecosystem health of Lake Winnipeg

As mentioned previously, the Lake Winnipeg Basin extends into four provinces and four U.S. states. The Freshwater Programs included $1.9 million to support governance activities in the basin. A key goal of the LWBI was to establish the Canada–Manitoba MOU as a foundation for a “long-term collaborative and coordinated approach between both governments, to ensure the sustainability and health of Lake Winnipeg and its watershed.” This agreement--the Canada–Manitoba Memorandum of Understanding Respecting Lake Winnipeg and the Lake Winnipeg Basin--was signed on September 13, 2010. An MOUsteering committee composed of federal and provincial representatives was established to guide activities under the MOU. Four principles direct and guide the actions of the parties under the MOU: openness and transparency; cooperation and collaboration; maximizing the benefits of existing resources or mechanisms; and accountability for activities. To date, two subsidiary arrangements--on science and on communications--are in the process of evaluation and negotiation.

In addition to the MOU, the governance element of the LWBI included exploring the need for an overarching governance mechanism to coordinate a basin-wide strategy for water management among various stakeholders. The Lake Winnipeg Basin Office, established under the LWBI, performs a communication and coordination function. However, no additional overarching mechanism was established. Rather, pre-existing governance mechanisms are utilized, facilitated by cross-membership or co-chair arrangements. LWBI key informants indicated that this approach was effective to ensure that the objectives and goals of various stakeholders and initiatives are commonly understood and efforts are aligned to remediate Lake Winnipeg. Here are some examples.

  • Environment Canada participated as an ex-officio member of Manitoba’s Lake Winnipeg Stewardship Board. The Board was established by Manitoba to make recommendations on how to reduce nutrient loading to Lake Winnipeg.
  • Environment Canada, along with Manitoba Water Stewardship, co-chaired a federal/provincial Lake Winnipeg committee, and a science sub-committee, intended to identify opportunities for collaborating and sharing information about respective programs and activities.
  • Environment Canada was a signatory to a multi-party working arrangement to foster transboundary coordination and collaboration for enhancing and restoring water quality in the Lake of the Woods watershed.
  • Environment Canada chairs the Prairie Provinces Water Board.
  • Environment Canada staff represent Canada as co-chairs for several International Joint Commission boards in the watershed, such as the International Red River Board, International Souris River Board and the international Rainy River Water Pollution Board. Environment Canada staff also co-chaired the IJC's International Lake of the Woods and Rainy River Watershed Task Force.
  • Stakeholders and decision-makers in the Lake Winnipeg and Lake Simcoe basins have the science-based information and tools to support water management activities and decision making for remediation and rehabilitation of areas affected by point and non-point sources of pollutants

Science was identified as a high-priority need in Lake Winnipeg to address an historical deficit in research and monitoring of the lake. Specifically, insufficient scientific knowledge was available to understand the ecology and nutrient cycling within Lake Winnipeg in order to establish relevant nutrient objectives and predict how the system would respond to management and policy decisions to control nutrients.

In 2005, a federal-provincial study on Lake Winnipeg submitted a dozen recommendations in the science area,26 leading to a provincial request that the federal government assist in addressing science needs and facilitate governance in the Lake Winnipeg watershed.

In total, $12.1 million in Freshwater Programs funding under the LWBIwas allocated to science. The Science Plan for Lake Winnipeg was developed to “facilitate the integration and coordination of science efforts across [the Lake Winnipeg] transboundary watershed in order to inform policy and programs, and support decision-making related to the nutrient management issues.” The following is a summary of the allocation of science funding across the fiscal years covered by the LWBI:

  • 2008–2009: $3.86 million
  • 2009–2010: $3.51 million
  • 2010–2011: $2.79 million
  • 2011–2012: $1.97 million

The Science Plan identifies six objectives, and science workplans are reviewed and updated annually. Four of these objectives refer to science-related objectives, while two refer to the dissemination and utilization of the research. The development of a single-window information portal is the key vehicle to promote data-sharing with key federal partners and ensure consistent and reliable delivery of relevant information. Science results support decision-making through such means as input, with the Province of Manitoba, into a framework on establishing nutrient objectives for Lake Winnipeg and performance indicators to monitor the state of the lake. A summary of activities according to the six objectives of the LWBI Science Plan is included in Appendix D (under separate cover).

In Lake Simcoe, science was acknowledged to be more advanced and, therefore, funding under the LSCUF did not include a dedicated science component. However, the terms and conditions of the contribution funding program did not preclude the funding of research and monitoring studies approved by the technical and public advisory committees.27 Thus, in Lake Simcoe, about one quarter (n=43) of projects funded under the LSCUF were research and monitoring projects. Funded projects contributed to knowledge gaps in the basin, notably in the areas of

  • sod farm runoff and tile drain loss of phosphorus in the Holland Marsh;
  • changes in sediment chemistry and biological communities over time and during different seasons;
  • levels and sources of atmospheric nutrients and pollutants;
  • identification of water quality parameters for recovery targets; and
  • research into significant plant growth triggers.

As well, O&M resources were allocated to conduct science work ($2 million). Results of research and monitoring activities are posted on the Department’s and project proponents’ websites and shared at science meetings and committees, and through publications.

Lake Winnipeg and Lake Simcoe – Intermediate Outcomes

  • Projects and knowledge lead to nutrient reduction and restoration of aquatic and fish habitat, and BMPs for nutrient management are developed, shared and used across the Lake Winnipeg and Lake Simcoe basins

Lake Winnipeg

The table below summarizes the types of projects funded by the LWBSF based on the sample drawn for the file review. Both the LWBSF and Lake Simcoe (discussed below) seem well positioned to make progress toward intended nutrient reduction and restoration outcomes as the majority of funded projects target these issues.

In Lake Winnipeg, 78% of projects received funding to reduce nutrient inputs from rural and urban sources. The LWBSF does not distinguish between point and non-point projects; however, non-point sources are estimated to make a greater contribution to the nutrient load in the lake compared to point sources.28

The same proportion of projects selected for the file review (78%) also had objectives related to the rehabilitation of priority aquatic ecosystems that support nutrient reduction and sequestration. These projects include, for example, landscape projects (tree planting, protection and stabilization of banks).

Table 3.5: Characteristics of Freshwater Programs Contribution Agreements: LWBSF
 Files (%)
Totaln=55
Reducing nutrient inputs from rural and urban sources78
Rehabilitating priority aquatic ecosystems that support nutrient reduction and sequestration78
Enhancing research and monitoring capacity to assist in decision making69
Controlling point and non-point sources of pollution54

While no targets were established for LWBSF funded projects, submitted proposals were assessed, in part, on the proposed project’s ability to achieve concrete outputs that would contribute to achievements in the program’s four priority areas. Using the Department’s management information system, the LWBSF tracks and aggregates outputs of funded projects using a number of indicators. For example, across seven projects completed at the end of 2009–2010, the following results were tabulated:

  • 530 head of livestock were restricted from surface waters;
  • 4,050 m of fencing were installed to restrict livestock access to surface waters and riparian areas;
  • 7,500 m of stream/lake bank were protected or stabilized;
  • 7,810 native plants, trees and shrubs were planted over an area of 8,886 m2; and
  • 75 km of land were surveyed to identify riparian rehabilitation and pollution sites.29

With respect to the development, sharing and utilization of BMPs, again, the program administrative data do not systematically track activity related to BMPs. However, the file review indicated that a number of projects were devoted to support for implementation of BMPs in the areas of livestock management, and sewage treatment and stormwater optimization guidance. Key informants characterized the program as making “reasonable progress” in this area, in collaboration with other organizations such as Agriculture and Agri-Food Canada. For example, a few interviewees noted that there is “good sharing of information--BMPs are well-known” and that LWBI has been useful to “get the ball rolling” with respect to implementation.

Nutrient reductions in the lake as a result of stewardship interventions--a more demanding indicator of program intended outcomes--are more challenging to monitor and assess. Improvements in nutrient loads, for example, may be difficult to detect due to normal fluctuations in nitrogen and phosphorous levels in the lake. Key informants noted also that current projects funded under the LWBSF represent the early days in restoration efforts, merely setting the stage for future dividends and are insufficient in and of themselves and at the current level of program activity to lead to detectable nutrient reductions.

Lake Simcoe

In Lake Simcoe, program-level data indicate that 96 of 163 projects funded (59%) were devoted to non-point source and habitat-related projects, 24 (or 15%) projects to point source and, as mentioned above, 43 (or 26%) projects related to research and monitoring.

Among the projects selected for the file review, 57% addressed rehabilitation of priority aquatic ecosystems that support nutrient reduction and sequestration. Note that point source pollution control projects in Lake Simcoe are not as common as other types of projects. About one third of selected projects (31%) had objectives related to rehabilitating priority habitats to restore the health of the aquatic ecosystem and coldwater fishery.

Table 3.6: Characteristics of Freshwater Programs Contribution Agreements: LSCUF
 Files (%)
Totaln=55
Reducing excessive nutrients from non-point sources in the Lake Simcoe Basin57
Enhancing research and monitoring essential to making progress in the restoration of the Lake Simcoe Basin and to assist in decision making43
Rehabilitating priority habitats to restore the health of the aquatic ecosystem and coldwater fishery in Lake Simcoe31
Reducing pollution from point sources in the Lake Simcoe Basin7

As in the case of the LWBSF, reporting for the LSCUFincludes aggregation of performance measures. Performance measures for completed LSCUF projects funded in rounds 1–4, include

  • (estimated) phosphorus diverted from use/reduced (138.57 kg/yr)
  • fencing installed to restrict livestock (6,612 m)
  • number of livestock restricted (506)
  • sediment release avoided (6.62 t/yr)
  • stream/lake bank stabilized (13,275 m2)
  • stream/lake bank stabilized (6.34 km)
  • erosion control structures (0.29 km)
  • environmental management plans (3)
  • native plants/trees plantings (27,321)
  • area of native plantings (370,806 m2)
  • manure storage facilities (4)
  • number of livestock whose manure will be managed (164 cows, 100 heifers, 16 calves, 26 horses)
  • in-stream structures (12)
  • aquatic habitat created (1.33 ha);
  • wetlands created (2.39 ha, 4000 m2)
  • shoreline habitat created (1.47 ha)
  • shoreline habitat preserved (2.0 ha)
  • habitat protected by land stewardship agreement (101.3 ha)
  • water-saving device (1)
  • septic system improvements (41)30.

With respect to the development and sharing of BMPs, the LSCUF has worked with the Ontario Ministry of Agriculture, Food, and Rural Affairs (OMAFRA) and the University of Guelph on agricultural BMPs. The program has focused on support for the implementation of BMPs(addressing financial barriers to implementation faced by landowners).

  • Stakeholder activities within the Lake Winnipeg Basin take into account their impact on the health of Lake Winnipeg

Assessing stakeholder activities within the Lake Winnipeg Basin on a systematic basis was beyond the scope of this evaluation and no program performance measurement information directly related to this outcome was available. However, key informants noted the following kinds of LWBI activities that engage stakeholders:

  • The LWBSF funds stakeholders within the Lake Winnipeg Basin to conduct activities related to nutrient reduction, point and non-point pollution control and habitat work. To date, there have been 29 project proponents under the LWBSF, leading 33 projects. According to file documentation, a total of 118 organizations participated in these LWBSFprojects.
  • The LWBI works closely with groups whose mandates or objectives relate to Lake Winnipeg, including, for example, the Lake Winnipeg Research Consortium. The LWBIco-hosts workshops to share understanding about the lake and restoration efforts.
  • The LWBI conducts outreach to the general public and others (e.g., private landowners). The Lake Friendly Campaign (funded in part by the LWBI)31 is an example of efforts to influence the behaviours of families, landowners and others to support clean waters in the Lake Winnipeg Basin. While the Lake Friendly campaign was not completed at the time of reporting, in a case study of the Lake Friendly campaign, project stakeholders indicated that the Lake Friendly website is evolving as a credible source of information in the community. Website statistics and external links to the site (1,995 links from other websites) suggest growing visibility of the campaign. In addition, the campaign has expanded to include the City of Winnipeg and to include a procurement component. Municipal, provincial and Crown corporations have participated in procurement programs that focus on products and services that are lake-friendly.

GLSRP – Immediate and Intermediate Outcomes

The intended outcomes of the Freshwater Programs pertaining to the GLSRP are

  • management of contaminated sediments in [eight] AOCs;
  • reduction of persistent toxic substances and other contaminants of concern;
  • restoration of beneficial uses;
  • contribute to the delisting of the AOCs; and
  • meeting Canada’s international commitments in the Great Lakes.

Each of these outcomes areas is discussed, in turn, below.

Management of contaminated sediments

Sediment remediation has been completed in one AOCs (Detroit River, Turkey Creek) and sediment management plans, including implementation of the selected sediment management strategy, are being implemented in two AOCs (Niagara River, Lyons Creek; and Bay of Quinte, Lower Trent River).

The sediment remediation projects in the five remaining AOCs are in various stages of site characterization, option assessment and engineering design. For these AOCs, the timelines and cost estimates that were developed at the outset of the GLSRP are no longer realistic. There are various challenges within these sites, such as unique site conditions that have required additional time to fully characterize the sites. Difficulties in securing funding from local stakeholders have also delayed option evaluation and engineering design. In several AOCs, there has been difficulty in securing funding from a community stakeholder and/or there has been a shift in the identified lead for the project. Appendix E (under separate cover) provides a summary of the status of sediment remediation projects in each AOCs.

Reduction of toxics and contaminants, restoration of beneficial uses and delisting of AOCs

Sediment remediation contributes to the reduction of toxics and contaminants and restoration of beneficial uses, which contributes to the delisting of the targeted AOCs. Note, however, that restoration and eventual delisting of AOCs is often dependent on numerous other actions in AOCs(e.g., physical landscape/watershed alteration, natural rehabilitation, wastewater upgrades). Current program data (included in Appendix F under separate cover), indicate that in the eight AOCs targeted for GLSRPfunding, a number of BUIs are assessed as having been restored (10 BUIs); however, many more remain impaired (although monitoring may not be up to date on the extent or nature of the impairments). While sediment remediation may be only weakly related to the restoration of many BUIs, sediment remediation is the final remedial action required in four of the targeted AOCs(St. Clair River, Thunder Bay, Peninsula Harbour and Hamilton Harbour), with delisting expected to occur quickly after this.

Meeting Canada’s international commitments related to Great Lakes AOCs

Canada’s agreement with the United States on the Great Lakes--the GLWQA--requires that both countries take remedial action in the heavily degraded AOCs. Remedial action plans (RAPs) are developed for all 40 AOCs (including 10 in Canada and 5 binational AOCs), with monitoring of progress conducted by the IJC.32 The IJC identified remediation and management of sediments contaminated with persistent toxic substances as a priority for restoration of Great Lakes AOCs in the mid-1990s. In its 13th Biennial Report (2006), the IJC noted that there were significant gaps in Canada (and in the United States) between the funds required and those appropriated to clean up contaminated sediment. Funding to the GLSRP was allocated to respond to these concerns. As noted above, progress toward the delisting of AOCs has been slow. In the eight AOCs requiring sediment remediation, none are currently delisted, although three are expected to be delisted by 2015.

3.2.2 Ultimate Outcomes

The Freshwater Programs are seeking to achieve two outcomes:

  • Lake Winnipeg Basin, Lake Simcoe and Great Lakes waters are clean and freshwater ecosystems healthy; and
  • Lake Winnipeg’s and Lake Simcoe’s contribution to economic activities, recreation and watershed functions are sustainable.

The program’s ultimate outcomes are, by definition, expected to occur in the longer term. Achievement of these longer-term outcomes is dependent on collaborations with other stakeholders, as well as ecological and environmental factors acknowledged to be outside the control of the Freshwater Programs. Clean water and healthy and sustainable ecosystems in the targeted lakes have not yet been achieved and, in fact, progress toward achieving these goals and the relative contribution of the Freshwater Programs will be difficult to measure. However, key informants viewed current efforts as laying the groundwork for these ultimate outcomes to occur. Improvements to the functioning of the watershed through stewardship projects (with a solid science and governance foundation), coupled with some of the aesthetic improvements to the lakes were noted as some early milestones toward these goals.

There are many factors external to the program that influence the achievement of intended outcomes, including the dynamic and complex nature of the ecosystem itself; multi-jurisdictional/ownership relationships that must be navigated (especially in the context of the AOCs); and economic constraints of stakeholders to share the cost burden of restoration.

Achievement of the Programs’ ultimate outcomes is supported by effective projects that are sustainable in the longer term. There is no evaluation evidence to assess the extent to which projects funded by the Freshwater Programs are sustainable. However, there were several examples of funded projects included in the case studies that were sustainable to some degree. These projects were sustainable through distribution and continued use of tools that were developed by the project (such as guidance manuals); science information used for decision making or in the development of BMPs; and the continuation of collaborations on freshwater issues (either continuation of the Freshwater project or new opportunities). Further, interventions that introduced physical changes to the environment--such as livestock fencing or removal of contaminated sediments in the Great Lakes AOCs--make an ongoing contribution to clean water.

Summary

Freshwater Programs have made progress in achieving many of the intended immediate program outcomes. The LWBI has made strides in addressing the deficit in scientific research available for the lake, as well as establishing an overarching governance mechanism with the province. Both the LWBI and the LSCUF have engaged stakeholders through their stewardship funding programs and through other outreach initiatives. While funded projects can point to output-related progress (e.g., habitats restored, plantings), it is much more difficult to measure outcomes at the intermediate level (e.g., nutrient loading in the lakes). Ultimate outcomes are expected to occur only in the much longer term.

For the GLSRP, progress on sediment remediation has occurred in only three AOCs (and two of these are in a longer-term process of monitored natural recovery). The majority of BUIs in these AOCs continue to be impaired and none of the AOCs have been delisted.

3.3 Economy and Efficiency of Program Design and Delivery

Overall Findings

While an RMAFwas prepared for the Freshwater Programs in 2008, there is currently no performance reporting at the program-wide level. The file review and key informant evidence indicates that the performance reporting for contribution agreement funding is adequate, though there are challenges in directly measuring and detecting changes in phosphorous levels and attributing these changes at the project level.

The evaluation evidence indicates that the design of the Freshwater Programs, which is unique for each of the three freshwater bodies, was generally viewed by key informants as appropriate, though a number of limitations in the current approach to stewardship, in particular, were noted. Complementary approaches involving other federal departments in areas such as regulatory initiatives or financial incentives were proposed.

Key informant and documentary evidence indicates that the allocated program resources for the GLSRP and LWBI may not be adequate to ensure that intended outcomes are achieved. Many activities and projects, especially in the Great Lakes, are difficult to predict with respect to estimated costs and timelines, and current estimates of project costs far exceed initial estimates. Further, the current funding terms and conditions for the GLSRP lack sufficient flexibility (e.g., in terms of leveraged funding requirements) to advance many of the remediation projects. Also, in the Lake Winnipeg Basin, current funding allocations are likely insufficient to effectively meet restoration objectives in the lake given the magnitude of the aquatic health issues and the complexity of the transboundary nature of this aquatic ecosystem. Funding levels for the LSCUF were considered adequate.

Roles and responsibilities appear to have been adequately defined and understood. The exception is Lake Simcoe where there were some initial challenges in clarifying the role of the public advisory committee and lack of involvement of the federal government in provincial governance mechanisms pertaining to Lake Simcoe.

Contribution program project proponents were satisfied with their experiences with the program, and offered some suggestions to improve the timeliness of the funding decisions and the mechanics of the application process.

The key informant and file review data revealed few unintended impacts of the program. However, some LWBSF and LSCUFproponents did identify unintended benefits such as enhanced relationships with collaborators, spinoffs into other projects and increased public interest or engagement in the issues their project and/or organization are addressing.

3.3.1 Performance Data

Evaluation Issue: Performance

5. Are appropriate performance data being collected, captured, and safeguarded? If so, is this information being used to inform senior management/ decision-makers?

Performance
Indicator(s)MethodsRating
Existence of effective performance measurement plan
  • Document review
Progress made, attention needed
Extent to which performance data is reliable, timely and relevant
  • Key informant interviews
Progress made, attention needed
Extent to which performance data informs/supports EC’s decision-making processes
  • Program document review
  • Program performance and financial data Analysis
  • Key informant interviews
  • Case studies
Progress made, attention needed

The Freshwater Programs’ Integrated Results-based Management and Accountability Framework (RMAF) (2008) defines ongoing program performance monitoring as “… the regular collection of information to help determine how the Program is doing at any point in time.” The information is to be used to manage the Program and to monitor and report progress toward achieving planned results. The key sources of information for monitoring and evaluation identified in the RMAFinclude project reports; a funding database; annual program reports; and general-public tracking research.33

The basis of performance reporting for the Freshwater Programs is project reports from contribution funding projects. Interim (quarterly) and final reporting is required of all LWBSF and LSCUFcontribution agreement proponents (with templates provided for both progress and financial reporting). Interim and final reports provide highlights of accomplishments; project activity; anticipated activity results (performance measures); activity status; and a description of progress to date (with an accompanying explanation of any changes from the project work plan). Final reports also provide some synthesis of the project experience, including challenges and lessons learned. Reporting of project accomplishments varies depending on the project objectives. For awareness or public education projects, data such as number of website hits, number of events, and number of attendees at workshops are presented. For habitat or point/non-point pollution source projects, data would include activity-based reporting such as the square metres of repaired water buffers, number of trees planted, and number of water samples/tests (these data were summarized in Section 3.2.1).

The file review confirmed that funded project documentation is quite complete--the majority of files contained documentation relevant to the purpose of the project and its relevance to objectives (proposal, proposal review / approvals), contribution agreement, interim reporting, financial records, a final report34 and associated products. About three quarters (74%) of the files included reference to the development of performance measures. Of those files, 90% provided evidence of relevant and timely performance data. The file review further confirmed that in the vast majority of files--92%--evidence of intended output achievement was provided to a very or moderately clear degree in the documentation. The files were far more likely to provide evidence of achievement of intended outputs than outcomes (69% clear or moderately clear evidence of outcomes). This is not unexpected given the projects are of a limited scope and duration, and outcomes related to water quality are difficult to detect and may materialize only in the longer term.

At the LWBSF and LSCUF funding program level, project reporting is captured in the Department’s management information system. This system (introduced midway through the Freshwater Programs) allows the Programs to record key information for all projects (tombstone information, funding/cash flow data, as well as project activities). Queries on the basis of these elements can be submitted for overall analysis and reporting. An interactive Web-based map of projects allows for broader public access to program activities.

There is no reporting that is Freshwater Programs-wide, due to the distinct nature of the three funding components. However, the Freshwater component programs conduct program-level progress/update reporting on a periodic basis (e.g., status reports on projects funded/funding allocated, aggregate measures of project outputs and accomplishments). For the LWBI science component, a science plan outlining the reporting requirements and six objectives and accompanying deliverables has been developed and is updated annually. LWBI program managers and staff noted that this regular planning and reporting exercise has been helpful to identify priorities and ensure progress against all six objectives.

For the GLSRP, individual project status reports are prepared and are periodically updated for the sediment remediation projects in each of the eight AOCs. A project status summary is also available that outlines project objectives and description, status report of essential elements or stages in remediation (e.g., problem characterization, options assessment, detailed engineering design), accomplishments, risks, and the proposed budget and timeline for the project.

In general, ECmanagers and staff indicated that the performance data and reporting that are available on the Freshwater component programs are useful to describe program activity, and highlight successful project implementation and the broader benefits of program investments to local communities. For contribution funding programs, the quality of the performance information is dependent on the quality of the final reports submitted by projects and use of the departmental reporting template which, when completed well, contributes to more accurate and comprehensive program-level reporting. Progress reports/updates are used internally for accountability purposes and course corrections during the program cycle.

Most funded project proponents feel that reporting requirements to EC are fair, appropriate and well-balanced between accountability requirements and level of effort. Several proponents noted a challenge in conforming to the departmental reporting templates, particularly the quarterly financial reporting template, which was described as cumbersome. This challenge is particularly difficult for projects with custom reporting obligations to multiple funders or for smaller organizations with little support staff resources.35

A specific measurement challenge that is inherent in initiatives to reduce lake-wide nutrients is detecting reductions in phosphorous in the lakes. Direct measures of phosphorous levels are subject to significant fluctuations in the lake, and therefore poor performance measures of restoration activities. To compensate for the lack of direct measures in phosphorus reduction, LWBSFand LSCUFstewardship projects are assessed on the outputs they achieved and on their potential to contribute to phosphorus reduction either directly (e.g., phosphorus diverted) and/or indirectly (e.g., reductions in phosphorous-bearing sediment released) (using a calculated equivalent of output x = x amount of phosphorous reduced). The precise contribution of individual projects to phosphorous reductions using this method is limited to the extent that actual phosphorous reductions may be affected by unique ecological factors in the specific area of watershed.

Summary

An RMAFwas prepared for the Freshwater Programs in 2008. The primary vehicle for performance reporting is the project reports from the GLSRP, LWBSF and LSCUF. For the GLSRP, sediment remediation project reports/project summary reports are prepared on a periodic basis to record progress. For LWBSFand LSCUFcontribution agreement funding, interim and final (narrative and financial) reports are required for all funded projects. Final reports more often indicate achievement of outputs, as opposed to outcomes, which are far more challenging to measure, especially given the short-term nature of the contribution agreements and challenges in measuring water quality changes in the lakes. Tracking of funded projects occurs through the Department’s management information system (introduced midway through the program). Annual program reports (at the Freshwater Programs level) were not submitted, although periodic update reports were prepared by each component program. Project proponent views of performance reporting requirements were mixed, with some experiencing difficulties with the financial reporting template in particular.

3.3.2 Program Design

Evaluation Issue: Performance

6. Is the Freshwater Programs’ design appropriate for achieving expected results?

Performance
Indicator(s)MethodsRating
Plausible link between program activities, outputs, and intended outcomes
  • Key informant interviews
  • Case studies
Achieved
Defined governance structure, including program processes, roles, responsibilities and accountabilities
  • Document review
  • Key informant interviews
Achieved
Freshwater Programs resources/capacity commensurate with expected program results
  • Document review
  • Program performance and financial data analysis
  • Key informant interviews
  • Case studies
Achieved
Views on the appropriateness of program activities, processes and governance structures
  • Key informant interviews
Achieved

Delivery Model

According to most key informants, the Freshwater Programs’ design and delivery is appropriate. A reported beneficial aspect of the Freshwater Programs is that the component programs are customized in their design based on the needs, issues and situation of each ecosystem. Key informants also agree that the activities and outputs of the three Freshwater Programs represent a sensible starting point in terms of contributing to the restoration and maintenance of the Lake Winnipeg and Lake Simcoe basins, and to sediment remediation in the Great Lakes.

The science-governance-stewardship approach (of Lake Simcoe and Lake Winnipeg) is a sound model, with the science component identified as a particular area of federal strength. According to some key informants, the nature of stewardship efforts in Lake Winnipeg and Lake Simcoe and the federal role in these efforts should be reassessed. Suggested improvements in this area and others are discussed in more detail in Section 3.4.3.

Resources

The internal resources and capacity of the Freshwater Programs were perceived to be adequate to implement program activities. For the LSCUF, a decision was made not to create a separate organizational structure or unit given the relatively short duration of the program (i.e., four years). Rather, the LSCUF group was housed in the same unit that oversees the Great Lakes programs. This was considered to be an advantageous management approach that leveraged internal expertise with nutrient and water quality and allowed for a rapid start-up of the program.

As mentioned above, the LWBI program resources were viewed by some key informants and external experts as insufficient to address the longer-term objectives of the program, given the size of the watershed and the seriousness of the nutrient issues to be addressed. The amount of program resources is expected to become increasingly problematic if the focus of the program moves from governance and science to action to restore the health of Lake Winnipeg. To implement restoration actions, allowing more directed funding to better target priority actions would be welcomed by current managers.

The current funding mechanism for the GLSRP has proven challenging over the first four years of the program. Six of the eight AOCs require that the federal government fund no more than one third of the total cost of the sediment remediation project.36 To date, sediment remediation work has been successfully negotiated and funded in three AOCs under these terms with provincial and community contributions: the Detroit River AOCs; the Bay of Quinte AOCs; and the Niagara River AOCs (Lyons Creek East, completed in fiscal year 2011–2012). These three GLSRP funded projects have leveraged funding between 50 and 84% of total project costs. The funding distribution is summarized in the following table.

Table 3.7: GLSRPLeveraged Funding
 ECProvinceCommunity StakeholdersTotalEC
Share (%)
Detroit River (Turkey Creek)$200,000$200,000$880,440$1,280,00016
Bay of Quinte* (Trent River)$200,000$200,000 $400,00050
Niagara River (Lyons Creek)$187,000$413,000$413,000$600,00031

* Inclusive of G&C and O&M.

For the remaining three AOCs where contribution agreement funding falls under the terms and conditions of the GLSF, securing matching funding in the one third (federal), two thirds (community) ratio has been challenging and, thus, progress in these AOCs has stalled. In these areas, community stakeholders such as municipalities cannot afford the level of investment required and industry is often financially unable or unwilling to contribute. In one AOCs, only the federal jurisdiction is relevant and so provincial, municipal or community contributions are unlikely to be leveraged.

Sediment remediation in the Niagara River and Hamilton Harbour is funded through departmental O&M dollars. O&M dollars were to be utilized for these two projects because other government departments were identified as the lead for these projects, thus requiring a transfer of funds (which can be accomplished using O&M funds, not contributions)37. In both projects, however, the originally identified lead department has shifted to EC. While the management framework allows for project leadership to be modified, the funding mechanism (O&M dollars) cannot. Thus, a mismatch has occurred between the type of funding that is available to fund sediment remediation work in these AOCs (O&M dollars) and the type of funding that is now required (contribution dollars).

Due to these difficulties, as well as the protracted development of sediment remediation plans in some AOCs, the program has reprofiled a significant amount of funds. A more flexible funding instrument that would avoid the need for reprofiling was viewed by key informants as preferable to the current GLSRP terms and conditions (an example of such a fund is the Federal Contaminated Sites Action Plan, which has a flexible cost-sharing ratio for cleanup projects and a 15-year timeframe to disburse the funds).

Roles and Responsibilities

Most key informants feel that the roles and responsibilities of those involved in the delivery of the Freshwater Programs initiatives are clear and generally understood by all relevant parties, particularly internal stakeholders. The understanding is viewed to have evolved and improved over the duration of the program. Note that the lines of authority for the Freshwater Programs are somewhat unusual in that they are dispersed across two branches within ECand two regions. Though not problematic or unusual, there is perceived to be a less comprehensive understanding of the roles and responsibilities of the program among external (to government) stakeholders.

In the Great Lakes, there are well-established mechanisms and processes to support restoration at the level of the AOCs, and roles and responsibilities are clear. The GLWQA (between Canada and the United States) and the Canada–Ontario Agreement are formal agreements that mandate and guide these mechanisms. Still, dealings with key stakeholders vary significantly across the AOCs due to different and often complex ownership of and authority over water lots (variously involving municipalities, Transport Canada, Port Authority, or Harbour Commission).

An early weakness in the LSCUF program was a lack of clarity in the roles and responsibilities of the PROPELcommittee (the public advisory committee formed by the Minister of the Environment). Several key informants noted that the structure and membership of the PROPELcame under criticism by some stakeholders who expressed concern about the credentials of the appointed members, transparency of the body and the potential for conflicts of interest. Work was required early in the program to clarify with members that the PROPELcommittee was an advisory, not a decision-making, body.

Most key informants feel that accountability for the Freshwater Programs and funded projects are enhanced through performance-reporting requirements. Further, the terms of reference for committees and consistent communication about roles and responsibilities to all parties were viewed as important to supporting clarity and accountability. Whereas in Manitoba, a formal agreement between the federal and provincial governments was put into place for the Lake Winnipeg Basin, there are no agreements between Ontario and the federal government regarding Lake Simcoe. Federal departmental representatives do not actively participate in the provincial governance of the lake.

Satisfaction with the Delivery of Contribution Agreement programs (Lake Winnipeg and Lake Simcoe)

The LWBSF and LSCUF funding application process was reported to be clear and adequately delivered by most project proponents. The application form was noted to be easy to follow and a few proponents noted their approval of the two-stage letter of intent proposal application process. The selection criteria were considered to be well laid out and became increasingly clear as applicants and EC staff completed subsequent rounds of project reviews and selection. Most proponents indicated that the support provided by EC for proposal development was helpful and that ECstaff were open to input on how to improve delivery.

Most key informants felt that the proposal review and approval process was appropriate, though a minority noted that information on the decision-making process was insufficient or that there was a lack of consistency in the decisions or how the funding criteria were applied.

A more common concern about program delivery was with respect to the timeliness of funding decisions. The delays in decision notification and execution of the agreement were frustrating for some proponents, especially in comparison to the relatively short timing between the call for proposals and closing date for submissions (typically only a few weeks), and particularly for those whose projects were seasonally sensitive.

The timeliness of funding decisions reportedly improved over time as experience was gained and review protocols became established. Internal respondents noted the challenges posed for technical and public advisory committees in reviewing a large number of proposals, as well as limited management and staff capacity to negotiate agreements (this last aspect being more prevalent during the early funding rounds). As well, for Lake Simcoe, the additional internal review and due diligence mechanisms associated with Ministerial-level approval tended to extend the timeframe associated with funding decisions.

The terms and conditions for the stewardship contribution funds indicate that the federal government will seek a cost-sharing ratio of from one third to two thirds from the provincial governments and other groups that propose projects to access contribution funding. This cost-sharing ratio seemed reasonable for most key informants. However, in Lake Simcoe, the amount leveraged from the province and other groups was not as high proportionately as departmental representatives would have liked due to economic constraints experienced by stakeholders (many of which were community-based organizations), and project proposals that assumed a two-thirds federal contribution. However, the higher federal contribution was perceived by key informants to have allowed a number of projects to occur that would not have otherwise. Also, some of these projects (e.g., native plantings, restoration of stream banks) included significant volunteer contributions that were not always accurately captured within the cost-sharing ratio.

Stewardship funding priorities in Lake Winnipeg and Lake Simcoe related to the reduction of phosphorus were viewed by most proponents as appropriate. Nonetheless, a small number of respondents offered the following suggestions to adjust the programs’ priorities in order to increase the Programs’ effectiveness:

  • increased targeting or strategic use of funds based on identification of needs in order to increase the benefits of the money invested
  • closer alignment of priorities with municipal objectives that are focused on infrastructure improvement, upgrading sewers and improving road drainage
  • inclusion of a fourth priority area for educational projects to encourage community change and adoption of practices that will result in phosphorus reductions at both point and non-point sources

Overall, project proponents offered the following suggestions on improvements to the application, decision-making, and funding processes:

  • Increase the time between the call for proposals and the closing date to afford organizations more time to prepare a response.
  • Afford project proponents the opportunity to present and respond directly to questions from the two committees (technical and public advisory) when proposals are under review.
  • Provide more information and clarity during the proposal process as to the level of reporting required (for example, have the reporting template online so that interested proponents could be informed of the level of effort involved).
  • Allow money to be reallocated across funding categories without requiring amendments to the contribution agreement (e.g., based on the discretionary approval of EC manager) in order to increase responsiveness to changes in project scope or timelines.
  • Locate project officers around the freshwater body in question to improve accessibility and build on the knowledge-base at EC by promoting a more localized understanding of the basin.

Summary

The design of the Freshwater component programs varies somewhat, but conceptually includes science-governance-stewardship elements.38 This approach was generally viewed as sensible to achieve intended program outcomes. The stewardship dimension could be enhanced to include other complementary approaches, given the scope of challenges in the lakes and more strategically focused to produce effects.

Program resourcing for the GLSRP and for the LWBIis problematic. For the Great Lakes, sediment remediation--particularly projects that require infrastructure investments--are difficult to predict with respect to estimated costs and timelines. In addition, the high cost of sediment remediation projects often present challenges for the province and community funders (municipalities, industry) to make a meaningful financial contribution. In six of eight AOCs, the federal contribution can be no more than one third the total cost of the project--a barrier to progress on sediment remediation projects in some of these AOCs.

For Lake Winnipeg, the funds allocated to the LWBI were substantially less than to the LSCUF, though the focus of the initiative was on advancing the science and governance in the lake. With progress evident on the science and governance fronts, and if the priority shifts to action to restore and maintain the lake, the current allocation is likely insufficient given the size of the watershed and the scope of the challenges that have been identified. No issues with respect to funding levels were identified for Lake Simcoe.

Program roles and responsibilities have been adequately defined and understood by those involved in the programs. The experience in Lake Simcoe was somewhat more mixed. The early experience with the public advisory committee required additional work to clarify the role of the committee. The federal-provincial relationship with respect to Lake Simcoe has been informal, to date, with the province being involved in the LSCUF technical advisory committee (though with no corresponding involvement of EC in the province’s efforts to protect Lake Simcoe).

The design and delivery of the stewardship contribution funding component of the Freshwater Programs is generally satisfactory from the perspective of proponents. A weakness in the delivery of these programs is the delay between proposal submission and funding approval. Delays in funding notification are particularly challenging for projects whose work is seasonal (e.g., water samples, planting).

3.3.3 Unintended Outcomes

Evaluation Issue: Performance

7. Have there been any unintended (positive or negative) outcomes?

Performance
Indicator(s)MethodsRating
Presence/absence of unintended outcomes
  • Program document review
  • Case studies
N/A
Views on whether unintended outcomes occurred
  • Key informant interviews
N/A

Unintended outcomes, when they were reported (by project proponents, in the file review, interviews and case studies), occurred most often at the project level and were generally positive in nature. Examples of unintended outcomes include a higher-than-expected number of collaborations formed and stronger-than-anticipated interest in project activities and the organization from citizens, landowners, volunteers and industry. Several projects noted spinoff benefits from projects: interventions to improve the health of the watershed functions (e.g., native plantings) had an unintended but positive impact on the aesthetic appearance of the lake and recreational use and enjoyment.

On the negative side, one proponent noted that some consultations took longer than expected and there was one mention of unexpected political and public criticism of funding provided to community projects. The Department’s response was to showcase benefits of the science investments and stewardship projects through public media (e.g., news releases, public service announcements).

Summary

Few unintended outcomes of the program were identified. Some projects did note unintended benefits of the project collaborations that were formed, which led to additional opportunities for working together, enhanced profile of their organization, and increased public interest or engagement in the issue.

3.4 Performance: Program Efficiency and Economy

Overall Findings

The file review data indicate that the LWBSF and LSCUF programs have achieved established targets with respect to cost-sharing ratios. The administrative efficiency of the LWBSF and LSCUFcomponents of the Freshwater Programs are in line with other contribution programs at Environment Canada. The GLSRPsediment remediation projects that are complete or nearing completion have been implemented under-budget and with significant leveraging from community funders. The Freshwater Programs’ science foundation, delivery with other collaborators, and knowledge transfer activities were seen to enhance efficiency. Efficiency improvements to contribution funding programs could be sought through more timely and streamlined administrative processes.

The evaluation indicated positive views of the cost-effectiveness of the Freshwater Programs generally, though some concerns were expressed about the absence of a strategic approach in contribution program funding priorities and criteria, and the efficacy of stewardship projects to address the serious ecological challenges in the lakes. Few more effective program alternatives were identified by key informants or in the literature. Among suggested improvements are granting to third-party delivery organizations or utilizing existing provincial/conservation authority funding mechanisms are alternative potential delivery mechanisms for disbursement of grants and contributions funding.

Lessons learned are being gathered and analyzed (both formally and informally) at the program component level. A number of themes emerged for suggestions for program improvements moving forward:

  • Stewardship activities should be realigned to ensure and strengthen cross-departmental/cross-jurisdictional engagement; include more strategic and/or pro-active approaches; and be reinforced through complementary policy and/or regulatory initiatives.
  • It was recommended that the program increase the clarity of the application of criteria used to identify priority aquatic ecosystems and priority issues addressed by the Freshwater Programs.
  • Resourcing should ensure the adequacy of funding relative to activities required to achieve the intended outcomes and should include sufficient flexibility in maximum federal funding requirements in aquatic areas where the federal government has primary responsibility or where there is insufficient alternative funding available.

3.4.1 Efficiency

Evaluation Issue: Performance

8. Are the Freshwater Programs undertaking activities and delivering products in the most efficient manner?

  • How could the efficiency of the Freshwater Programs activities be improved?
  • Are there alternative, more efficient, ways of delivering the Freshwater Programs?
Performance
Indicator(s)MethodsRating
Comparison of program activities and products delivered by other similar programs
  • Document review
  • Program performance and financial data analysis
  • Key informant interviews
  • Case studies
  • Literature review
Achieved
Resources leveraged from contributions and their perceived impact on the Freshwater Programs
  • Document review
  • Program performance and financial data analysis
  • Key informant interviews
  • Case studies
Achieved
Analysis of actual program operational costs in relation to the production of outputs
  • Program performance and financial data analysis
  • Project-level file review
  • Case studies
Achieved
Views on whether the cost of producing program outputs is as low as possible
  • Key informant interviews
Achieved
Views on how the efficiency of program activities could be improved
  • Key informant interviews
Achieved
Views on whether there are alternative, more efficient, ways of delivering the Freshwater Programs
  • Key informant interviews
Achieved

This section examines the efficiency of the Freshwater Programs’ activities and delivery--whether the programs are undertaking activities and delivering outputs at least cost. Evidence of program efficiency is drawn from analyses of administrative costs for the LWBSF and LSCUFcontribution programs, as well as feedback from key informants. Note that program leveraging, presented in Section 3.2.1, is often identified as an indicator of efficiency.

It is difficult to determine the cost-efficiency of GLSRP, based on its current progress only midway into its program life cycle. The limited evidence that is available suggests that there has been efficient delivery of sediment remediation projects in the three AOCs where work has been completed or is nearing completion. The work has been completed under budget and projects have leveraged funding exceeding the minimum that is required. Work in the remaining five AOCs has not yet advanced to a stage of negotiation where leveraging can be examined and the program has reprofiled much of the EC budget intended to fund the work.

A common indicator of program efficiency is administrative efficiency (the ratio of operational costs to program dollars), which can be computed for the LWBSF and LSCUFcontribution programs under the Freshwater Programs. For the LWBSF, in total 44 projects were funded over the 4 years of the program, with the average project contribution being $66,796. Over the same period, for every contribution dollar spent under the LWBSF, $0.22 was spent on salaries and O&M. For the LSCUF, 156 projects with an average project contribution value of $97,710 were funded during the first 3 years of the program. The overall administrative ratio for the 4-year period was $0.15 for every contribution dollar spent.

Table 3.8: Lake Winnipeg Basin Stewardship Fund and Lake Simcoe Clean-Up Fund Cost-Efficiency Analysis

Program: Lake Winnipeg Basin Stewardship Fund and Lake Simcoe Clean-Up Fund cost-efficiency analysis.
Type2008–20092009–20102010–20112011–2012*
O&M$77.4K$52.9K$52.9K$53.9K
Salaries$138K$107K$107K$53.5K
Grants and Contributions funding$0$1,085.5K$958.5K$895K
Number of projects funded0141911
Average funding / project--$77,536$50,447$81,364
Average (O&M+ salaries) / project funding--0.150.170.12
Overall administrative ratio (4 years)0.220.220.220.22

* LWBSF 2011–2012 data reflects budgeted spending.

Program: Lake Winnipeg Basin Stewardship Fund and Lake Simcoe Clean-Up Fund cost-efficiency analysis.
Type2008–20092009–20102010–20112011–2012
O&M$417.8K$362K$437.7K$39.7K
Salaries$186.8K$365.3K$325.7K$132.9K
Grants and Contributions funding$1,862.6K$6,005.8K$6,991.9K$382.5K
Number of projects funded615243--
Average funding / project$30,534$115,496$162,023--
Average (O&M+ salaries) / project funding0.320.120.11--
Overall administrative ratio (4 years)0.150.150.150.15

The administrative ratios for the LWBSF and LSCUF are comparable to other EC grants and contributions programs and, at $0.22 and $0.15 for every dollar in contribution funding allocated, the ratios are situated at the mid-range of the spectrum. For example, the Great Lakes Sustainability Fund was calculated to have an administrative ratio of $0.24. Both EC’s Habitat Stewardship Program and Invasive Alien Species Partnership Program had administrative efficiency ratios of $0.13.39

The views of key informants on the perceived efficiency of the Freshwater Programs overall, and at the project level, are positive. Among the key factors that are seen to be contributing to the Programs’ efficient delivery is their science foundation. Specifically, the federal scientific and technical expertise was noted to be valuable in understanding and assessing the technical aspects of funding proposals. In addition, scientific data was perceived to be extremely helpful in identifying priorities, and establishing a link between restorative actions and priorities.

Another aspect of the programs that was reported to contribute to efficiency is the leveraging of resources from other stakeholders. According to the project file review data, federal funding constitutes, on average, about 40% of the total project cost of contribution funded projects, and all projects leveraged monetary (87%) or in-kind (98%) contributions from the proponent organization or collaborator. Most project proponents indicated that without the federal funding their project would either have not gone ahead or the scope of the project would have been significantly reduced. In particular, these contributions are seen as extremely valuable as they create connections with grassroots or community-based organizations that possess expertise, regional networks, community and geographic reach and volunteers. Specific comments to this effect from key informants include the following: “their drawing on the expertise of [organizations] with experience on the Lake was critical”; and “It is very efficient… through building on existing networks, committees, institutions already in place and interested in Lake Winnipeg.”

Finally, according to program key informants, efficiency is being encouraged by the transfer of experiences and best practices as the programs proceed, and adapting the programs based on lessons learned. The transfer of research and knowledge (e.g., through the information portal in Lake Winnipeg or through the compilation of lessons learned in Lake Simcoe) also encourages efficiency. At the project level, the case studies provide many examples of project investments where there are sustainable, ongoing impacts on watersheds (e.g., fencing) and knowledge transfer within the province (e.g., guidelines for sewage treatment plants).

Key informants were also asked to identify any factors that undermine program efficiency or aspects of the program that could be adjusted to improve efficiency. One such factor, particularly in the context of the GLSRP, is the complexity of negotiations with key stakeholders. Key informants reported that, in some communities, progress is stalled because municipalities are financially constrained or corporations are unable or unwilling to address historical sources of contamination.

With respect to contribution programs, there are concerns that the bureaucracy associated with the programs’ selection and approval processes is leading to delays that ultimately hamper the efficiency of the Programs’ work. Specifically, there is a considerable delay in getting from the proposal review stage to contribution agreement signature. Delays in approvals create challenges for projects to get up and running quickly and to complete their project in a timely way. Related to this was the rapid start-up of the Freshwater Programs itself, initially marked by lack of clarity around program goals and priorities and limited tools and resources for program delivery.

Lastly, at the project level, unexpected individual circumstances, such as those related to human resources (e.g., turnover) and weather were identified as hindrances to efficient project delivery.

Summary

The administrative efficiency of the LWBSF and LSCUFcomponents of the Freshwater Programs are in line with other contribution programs at Environment Canada. Supporting program efficiency are the Programs’ science foundation; leveraging monetary and in-kind contributions from an array of collaborators (and other contributions such as expertise, volunteer hours, networks); and efforts to share information and lessons learned based on program experience. Potential improvements to the efficiency of program delivery would include a streamlined application process.

3.4.2 Economy

Evaluation Issue: Performance

9. Are the Freshwater Programs achieving their intended outcomes in the most economical manner?

Performance
Indicator(s)MethodsRating
Extent to which program-intended outcomes have been achieved at the least possible program cost
  • Document review
  • Program performance and financial data analysis
  • Case studies
Progress made, attention needed (~)
Views on whether good value is being obtained with respect to the use of public funds
  • Key informant interviews
Progress made, attention needed (~)
Evidence of/views on whether there are alternative lower-cost program models to achieve the same expected outcomes of the program
  • Document review
  • Key informant interviews
  • Case studies
  • Literature review
Progress made, attention needed (~)

The issue of economy revolves around whether program outcomes are achieved at least cost and whether there are alternatives to the current program that could achieve the same outcomes at a lower cost. In this section, the evaluation findings presented include the perceived cost-effectiveness of the Programs and key informant opinions on alternative, more cost-effective approaches to program delivery.

There was agreement among the majority of the key informant respondents that the Freshwater Programs are generally the most cost-effective means of restoring and maintaining the health of targeted aquatic ecosystems. The science and governance aspects of the programs were seen as particular strengths. The programs have leveraged funds from the province and community organizations to a significant degree. As well, involving local authorities and grassroots groups in stewardship projects has also leveraged their on-the-ground knowledge. Community connections were also reported to contribute to the cost-effectiveness of the program. The projects that were profiled in the case studies were also notable in terms of delivery with other collaborators: case study respondents frequently highlighted the importance of the federal funding in building the credibility of the project and their ability to successfully negotiate relationships with other stakeholders. Suggestions to improve program efficiency and economy were canvassed and are consolidated in Section 3.4.3.

There were few program alternatives at the Freshwater Programs level identified by key informants. The review of international literature also did not reveal any alternative, more effective models. A brief review of the experience in the area suggested that while there were significant variations in the overarching freshwater policies implemented across the six jurisdictions studied, at some level, they all had implemented policies, strategies and frameworks that included broad, shared principles relating to themes such as watersheds as organizing units; sustainability; stakeholder involvement; economic instruments; user-pay models; and the integration of water with other sectors.

There were some mentions of alternatives to the current federal approach to stewardship funding. This could include, for example, transferring the delivery of funding for community-based stewardship projects to another funding mechanism similar to those available at the provincial level or through local conservation authorities.

Summary

Few alternative approaches to the Freshwater Programs are available at this time, though there may be some more economical methods to distribute funds for stewardship (e.g., through larger grants to third parties or existing funding mechanisms at the regional or provincial level).

3.4.3 Lessons Learned and Suggestions for Improvement

Evaluation Issue: Performance

10. What lessons have been learned at the program and/or initiative levels? Have they been used to improve program performance and/or results-based management? Have they been instructive to the design and/or improvement of other programs?

Performance
Indicator(s)MethodsRating
Evidence of lessons learned being explicitly gathered and analyzed
  • Document review
  • Key informant interviews
  • Case studies
Achieved
Evidence of program improvements coming from the lessons learned
  • Document review
  • Key informant interviews
  • Case studies
Achieved
Evidence of lessons learned being transferred to other similar programs
  • Key informant interviews
  • Case studies
Achieved
Evaluation Issue: Performance

11. What are the most important opportunities for improvement going forward with the Freshwater Programs?

Performance
Indicator(s)MethodsRating
Views on the most important opportunities for improvement going forward with the Freshwater Programs
  • Key informant interviews
N/A

Key informants and case study respondents were asked to describe any lessons learned that have emerged from their experience with the Freshwater Programs or its component initiatives. Lessons that were identified included

  • the importance of having the tools in place to manage funding and to communicate funding opportunities broadly and quickly in order to support rapid program start-up when funding is announced;
  • the value of collaborating with other stakeholders at the program and project level--thus facilitating coordination of efforts (including enhancing communication among the stakeholders, sharing information, and establishing buy-in) and leveraging of dollars, expertise (such as in research) and reach (such as in local knowledge or the resources to carry out a geographically disperse project); and
  • the benefit of analysis of the external environment (e.g., current research and restoration programs/activities, strategic direction of other jurisdictional stakeholders, governance framework within the freshwater ecosystem) to inform program development. For example, in 2008, the Ontario government enacted the Lake Simcoe Protection Act (LSPA) and launched a comprehensive protection plan in 2009. If the intent of the provincial government with respect to Lake Simcoe had been identified at the design phase, the LSCUF activities could have been better aligned to leverage provincial activities under the LSPA and the protection plan.

Both Lake Winnipeg (informally) and Lake Simcoe are compiling lessons learned reports based on program experiences to date to inform future programming decisions should the Freshwater Programs funding be renewed. Because the GLSRP is only midway into its program cycle and activity has been modest, lessons learned have not yet been synthesized.

When asked to identify what they believed to be the most important opportunities for improvement going forward with the cleanup and maintenance of healthy freshwater ecosystems, key informants had a number of suggestions. Consistent with a theme identified in earlier comments, many key informants addressed their suggested improvements to the stewardship dimension of the Freshwater Programs delivery model. Some key informants questioned the effectiveness of project-based stewardship initiatives to address the serious and persistent nutrient challenges in the lakes and a few also questioned the appropriateness of the federal level in this type of effort, which is generally coordinated at a more local or regional level. Proposals to re-conceptualize this aspect included

  • complementary efforts to strengthen policy (e.g., incentives for implementation of BMPs, enhanced compliance with existing regulations) that may have a more significant and sustainable impact on the lakes; and
  • broader engagement of other government departments that have regulatory authority to enhance coherence of programming. This could include other Canadian federal departments (e.g., AANDC, AAFC, DFO), or perhaps collaborations between provinces (Ontario and Manitoba) or with the United States.

It was also suggested by a few departmental and external expert key informants that a review of the criteria used to identify priority areas and issues addressed by the Freshwater Programs (and their application) may be appropriate to better inform future program design and funding decisions. Notably, there was a sentiment that the Department should clarify the criteria and process used to determine which freshwater ecosystems should be targeted and what activities should be prioritized for each selected ecosystem. Note that since the Freshwater Programs were established, EC has developed the Priority Ecosystem Initiatives Management Framework (PEIMF), which seeks to identify and prioritize actions in Canada’s ecosystems.

In Lake Winnipeg and Lake Simcoe, some key informants recommended adjusting the contribution funding programs’ scope or priorities. These comments took various forms and included the following suggestions:

  • Ensure funding allocated to projects is more strategic and grounded in scientific research (e.g., addressing main contributors to nutrient loading, determining the kinds of projects that are making the greatest difference). More narrowly defined funding priorities/criteria, or having a pool of directed contribution funding could be a way to enhance the proactive and strategic use of funds.
  • Adopt a more holistic approach to the stewardship of the lakes that would include attention to issues beyond nutrients (e.g., urban stormwater, invasive species).
  • Focus on more proactive approaches, such as communications and outreach activities that promote a greater awareness of the challenges in the lakes and ensure broader citizen engagement in order to influence individual behaviours.
  • Provide sufficient monitoring of projects identified as potential demonstrations and dissemination of their results.
  • Communicate the local benefits of stewardship projects to stakeholders and to the public.

Finally, key informants from GLSRP and Lake Winnipeg indicated the need for improvements to the resourcing of the Freshwater Programs. Specifically, there is perceived to be greater flexibility needed for GLSRPfunding, particularly the barrier to progress posed by the maximum one-third federal funding threshold. In Lake Winnipeg, if a future program moves toward restoration activities, the level of funding should be re-evaluated (the lake is a “big area with big issues”). Greater local availability and participation of departmental scientific and funding program staff was also recommended by project proponents.

Summary

Both Lake Winnipeg (informally) and Lake Simcoe are compiling lessons-learned reports based on program experiences to date to inform future programming decisions should the Freshwater Programs funding be renewed. A number of program improvements were suggested pertaining to

  • enhancing the stewardship element of the programs’ design to increase efficacy;
  • rethinking the selection of priority freshwater systems for funding,
  • improving the contribution funding process to include more strategic and proactive priority-setting; and
  • adjusting program resources to better meet program needs.

12 Office of the Auditor General, 2008 March Status Report of the Commissioner of the Environment and Sustainable Development.

13 State of the Great Lakes 2009, Highlights, Ottawa and Washington: Governments of Canada and the United States of America, Environment Canada and United States Environmental Protection Agency, 2009.

14 International Joint Commission, Fifteenth Biennial Report, 2011.

15 Progress Report on the Lake Simcoe Clean-Up Fund 2008–2009, Environment Canada, 2010.

16 Minister’s Annual Report on Lake Simcoe 2010, Ministry of the Environment, 2011.

17 Minister’s Annual Report on Lake Simcoe 2010, Ministry of the Environment, 2011.

18 Water quality trading is a market-based “cap and trade” system to reduce the cost of water quality improvements (e.g., in the Lake Simcoe watershed, the system would allow (more expensive) point pollution sources such as sewage treatment plants to offset their phosphorus emissions through trade with farmers to reduce their non-point phosphorus discharges (less expensive).

19 Alberta, Saskatchewan, Manitoba, Ontario, Montana, North Dakota, South Dakota and Minnesota.

20 Lake Winnipeg Research Consortium Inc.

21 Bunting. L., Leavitt, Sudden Ecosystem State Change in Lake Winnipeg, University of Regina, 2011.

22 Options included government, water companies, large companies, citizens, farmers and NGOs.

23 The LWBI science and governance streams, which are incorporated under the Water Resources program activity (PAA 1.2).

24 In this framework and subsequent analyses, the Great Lakes are among the high-ranking vulnerable ecosystems identified nationally and recommended for coordinated action.

25 Office of the Auditor General, 2010 Fall Report of the Commissioner of the Environment and Sustainable Development.

26 Restoring the Health of Lake Winnipeg, Canada’s Sixth Great Lake, A Report by the Lake Winnipeg Implementation Committee, November 2005.

27 The terms and conditions of the Freshwater Programs include as eligible activities “costs to undertake ecosystem status, assessment and reporting to aid implementation of stewardship and remedial actions to achieve program objectives.”

28 Point sources in Manitoba are estimated to contribute 5% of total nitrogen and 9% of total phosphorous to the lake compared to 24 and 32%, respectively, from non-point sources.

29 Progress Report on the Lake Winnipeg Basin Initiative, 2008/09 – 2009/10.

30 Completed LSCUF Projects (Rounds 1-4).xls

31 The total project value was noted to be $490,432, of which $252,952 was funded by the LWBSF. The objectives of the Lake Friendly Campaign are threefold: to reduce the nutrient load entering Lake Winnipeg; to encourage environmentally responsible consumer choices to improve water quality in Lake Winnipeg; and to build consumer awareness of environmental issues important to improving the health of Lake Winnipeg.

32 Using the RAP reporting process, Canada and the United States as parties to the agreement are directed to classify Areas of Concern by their stage of restoration progressing from the definition of the problems and causes (Stage 1 RAP), through the selection and implementation of remedial measures (Stage 2 RAP), the monitoring of recovery, and when identified beneficial uses are no longer impaired and the area restored, the removal of its designation as an Area of Concern (Stage 3 RAP).

33 Note that funding for the Freshwater Programs included $1.76 million for a communications component, which was allocated to a separate unit within Environment Canada in order to undertake communications activities in support of the Freshwater program.

34 Note that the files that were reviewed for Lake Winnipeg included some projects that were complete, but awaiting the final report.

35 Note that a new departmental G&C reporting framework that links reporting requirements with an assessment of risk was introduced at Environment Canada midway through the program. The framework aims to reduce the reporting burden for lower-risk projects. The impact of the new framework was not assessed directly through the evaluation.

36 Sediment remediation in these six AOCs(Detroit River, St. Marys River, Thunder Bay, Peninsula Harbour, St. Clair River, and Bay of Quinte) is funded by contribution agreement utilizing the GLAP GLSF terms and conditions, which allows for a maximum of one-third federal funding of the costs of remediation.

37 For example, in the Niagara River, Transport Canada owns 98% of the affected area; however, it has not informed the technical advisory group in this AOCs of its plans for this area.

38 This is less true of the GLSRP, which is focused on remediation of contaminated sediment. Science and governance aspects of this program occur under the Great Lakes Action Plan (GLAP).

39 GLSF is a 5-year $12.7 million Grants and Contributions program, with an average project cost of $57,500. Per-project costs were not publicly available for the Habitat Stewardship Program or the Invasive Alien Species Partnership Program, though total funding available for these programs is $37 million (over 4 years) and $5 million (over 5 years) respectively.

Date modified: