Final Report: June 10, 2010
Previous page | Table of Contents | Next page
The evaluation of Environment Canada's GLAP IV focused on the five-year span of the program, from fiscal years 2005‑2006 to 2009‑2010. The findings of the evaluation lead to the following broad conclusions about the relevance, design and delivery, and performance (effectiveness, efficiency and economy) of GLAP IV.
1) The evaluation evidence indicates that GLAP IV is an important program to address federal commitments under the GLWQA and COA with respect to Great Lakes AOCs. The program is needed to continue to address the complex and evolving Great Lakes ecosystem. Public opinion supports the continued need for the program, as Canadians indicate water resources to be a high priority. The work toward delisting of AOCs is not yet complete and, in light of finite resources, the use of the AOC designation is a way to focus resources and attention (with a science-based framework and including local involvement) on these highly degraded areas that impact the overall quality of the Great Lakes. The GLAP IV program priority areas continue to resonate and do not require adjustment.
2) GLAP IV is the only program that focuses on Great Lakes AOCs, acting as a catalyst for leveraged funding from other sources. While there are other sources of funding that support remediation as well as science and monitoring in the AOCs, there was no evidence of risk of duplication or overlap. This risk is mitigated by coordination at the program‑officer level and through collaborative networks of provincial, federal and U.S. scientists and technical experts.
3) GLAP IV aligns well with federal and departmental priorities. At the federal level, a "healthy environment for Canadians," including clean water, is a stated priority, and at the departmental level, GLAP IV is consistent with Environment Canada's mandate to preserve and enhance the quality of the natural environment. Moreover, the Great Lakes are identified as a high‑priority ecosystem nationally and for the Department.
4) The shared boundary of the Great Lakes waters with the United States and Canada's commitments under the GLWQA provide a strong rationale for federal jurisdiction in the AOCs. As well, a number of legislative authorities (the International Boundary Waters Treaty Act, Clean Water Act and Canadian Environmental Protection Act, 1999) support federal jurisdiction in this area.
5) The GLAP IV model is a sensible approach to achieve program outcomes. Engagement of partners and involvement at the local AOC level are distinguishing and positive features of the model that leverage expertise and resources for remediation. The overarching framework for guiding GLAP IV and the science-based approach are also notable strengths that organize efforts in the AOCs and maintain the focus on delisting.
6) Delivery of some aspects of GLAP IV is inconsistent with the original design. The annual work planning process for the program and aspects of the program's management framework were not implemented. Therefore, few levers were available to the program to monitor and ensure that the activities to achieve the goals of GLAP IV, and the activities for which partners were funded, were undertaken in a coherent way. The distributed responsibility model was an identified weakness in other reviews of the program, with a recommendation to ensure that appropriate resource allocation, capacity and priority are assigned to meet program goals. Evidence suggests that many or all of these issues stemmed from the departmental transformation that occurred in 2005‑2006.
7) Resources for funding federal and GLSF partners to undertake priority activities in AOCs are largely sufficient, with increases needed to address inflation as well as the declines in funding to the GLSF that followed the first year of GLAP IV. The effectiveness and efficiency of departmental partners' efforts was attenuated by a limitation on the ability of Environment Canada programs to convert O&M to salary dollars, which created challenges in hiring the human resources required to achieve project objectives.
8) Financial accountability of the program was weak. Following transformation, program resources were no longer tracked by program but rather by departmental result. Furthermore, all B-base funding (i.e., temporary funding received for a specific purpose) was unbundled at this time, and so financial tracking of GLAP IV funding at the program level by internal Environment Canada partners was very limited. Some federal partners may not have uniformly received the GLAP IV funds that were allocated for their approved work plans.
9) Existing processes for reporting of GLSF-funded projects are quite complete (though only entered electronically to 2006‑07) and are well-understood by project proponents and managers. Reporting is focused to some extent on achievement of outputs (as opposed to outcomes, which are far more challenging to measure). For projects conducted by federal partners, processes for reporting to the program are not clearly articulated and have been sporadic over the life of the program.
10) Conducting monitoring and measuring results for program performance purposes is weak. Program reporting is not currently guided by a performance measurement framework and no systematic process exists to aggregate or summarize project-based reporting (costs [including partner contributions], activities and outputs).
11) The design and delivery of the GLSF component of GLAP IV is generally satisfactory. The program is longstanding and many proponents are involved in ongoing projects under this program. GLSF funds are an important catalyst in leveraging funding from other sources. The deficiencies in delivery of the program are the delays between proposal submission and funding approval (due in part to the program transition from the use of MOUs to contribution agreements as part of a Department-wide initiative to increase accountability), and the complexity of the application process. As several AOCs are moving closer to delisting, developing delisting criteria and clarifying responsibilities and the process for delisting are also emerging issues.
12) GLAP IV has made solid progress in achieving shorter-term outcomes, but only modest progress toward long-term outcomes that build on the achievements of predecessor GLAP programs. Only three AOCs have been delisted and one AOC designated an area in recovery, and these occurred prior to GLAP IV. Moreover, a majority of the BUIs identified in 2003 remain impaired. The original program goal of completing federal actions in seven group 1 AOCs has not been achieved. Still, momentum appears to be increasing as the program expects that three group 1 AOCs will be delisted within two years. As well, a host of external factors influence program effectiveness, including the dynamic and complex nature of the ecosystem, the program's dependence on contributions of other partners to achieve its outcomes, and significant challenges in developing delisting criteria for BUIs that must be addressed.
13) Unintended impacts of the program are few in number but positive in nature, such as unanticipated public interest, engagement of partners and technical transfer.
14) Although data related to administrative expenses for the first four years of the program were available for the GLSF component only (with administrative costs for 2007‑2008 based only on program estimates), these limited data indicate that efficiency of this component compares reasonably well with other contribution programs at Environment Canada that require more intensive staff involvement in program delivery. There are, however, a number of avenues to improve efficiency at the program level (e.g., improve strategic management, and coordination among federal partners) and at the GLSF project level (e.g., streamlined application process).
15) GLAP IV is an appropriate investment of public funds. Results are beginning to be achieved in the AOCs, though time frames for delisting have been long. As well, there are some broader economic benefits to restoration of AOCs. There were no alternative approaches that would result in a more economical achievement of program objectives.