Formative Evaluation of the Federal Contaminated Sites Action Plan

Final Evaluation Report

| ToC | Previous | Next |


4.0 Conclusions and Recommendations

This section concludes on the findings of the evaluation and makes recommendations for management to consider.

4.1 Design and Delivery

Governance

The evaluation concluded that the governance structure of FCSAP is appropriate and the roles of the FCSAP Secretariat, Expert Support, Custodians, and CSMWG are clear and well understood.  Two deviations from the original plan for FCSAP have occurred.

First, the bi-annual meetings of the FCS Steering Committee, which were intended to provide oversight and strategic direction for FCSAP, have become a venue where the operational issues of FCSAP are discussed by DGs and Directors who attend in place of ADMs.  The meeting agendas, set by the FCSAP Secretariat, and the formal nature of the meetings are not conducive to raising and discussing strategic issues.  Despite the low ADM attendance at Steering Committee meetings, the evaluation did not find any evidence that the program has suffered.

Second, a Third Party Expert Panel whose purpose was to provide the program with an external perspective and independent advice has not been implemented.  The evaluation found that no key strategic issues have yet arisen for a panel to discuss. However, there might be merit in seeking expert external advice when planning the post-2010 period of FCSAP.

Recommendation 1: The FCSAP Secretariat, in cooperation with the Treasury Board Secretariat, Real Property and Materiel Policy Division, should consider adjusting the FCS Steering Committee from an ADM level to a DG level committee.  The discussion of strategic issues should be a regular agenda item for Steering Committee meetings.  

Recommendation 2: The FCSAP Secretariat should consider seeking the advice of external experts when developing strategies for the post-2010 period of the program.

Allocating Resources and Addressing Risk

The evaluation found that the program has well-documented procedures, standards for assessing risk, a risk ranking methodology, and other related guidance documents produced by experts in the field.  In general, resources are allocated to the projects of highest priority.  Factors such as external influence and risk assessments conducted by inexperienced consultants can cause resources to be misallocated, but these instances are rare.  

The evaluation revealed that Custodians have tended to err on the side of caution and proceed with remediation when more cost effective risk management approaches could have been pursued.  To assist Custodians in making these difficult decisions, the FCSAP Secretariat and a consultant are currently developing a Decision-Making Framework that is slated for release in March 2009.      

Some FCSAP projects are particularly large and utilize a considerable share of resources.  These projects pose considerable risk to the program.  While the progress of a few projects has been peer reviewed (e.g., Faro Mine), most have not.  

Recommendation 3: For very large projects, the FCSAP Secretariat should consider requiring peer review of remediation/risk management plans and progress.  External experts or interdepartmental representatives could be drawn upon to undertake the peer review.  Large projects could be considered those in excess of a certain dollar level of liability or ranking score.  

There are other initiatives that FCSAP could pursue to ensure that resources are utilized effectively.  A remediated site that was heavily contaminated or that used an engineered approach such as constructing landfills, dams, or dumps may continue to pose risks.  For instance, climate change may compromise the functioning of a frozen landfill in future years.  At present, FCSAP does not require these sites to be monitored.

Recommendation 4:  The FCSAP Secretariat, in cooperation with the Treasury Board Secretariat, Real Property and Materiel Policy Division, should consider making long term periodic monitoring mandatory for the most contaminated FCSAP sites.  

To reduce risk, FCSAP should consider adopting a number of practices encouraged by certain provinces.  Currently, the closure reports on provincial sites required by the provinces of British Columbia, Ontario, and Quebec provide useful summaries of the original problem, the solution pursued, and any remaining risks or concerns that should be noted.  In addition, some provinces place a restrictive covenant or other instrument on land title identifying the risk-management requirements and/or areas.  This mechanism ensures that any new property owner is aware of the contaminated area and his/her responsibilities for risk managing and monitoring.  

Recommendation 5: The FCSAP Secretariat should consider making project closure reports mandatory for all FCSAP projects.  

Recommendation 6: The Treasury Board Secretariat, Real Property and Materiel Policy Division, should consider making restrictive covenants on land titles mandatory for FCSAP sites that are being risk managed or monitored.

Program Funding and Human Resources

The evaluation found that funding for projects is adequate in most cases. The exceptions are DND, whose demands for assessment funding often exceed the amount that the program can provide, and DFO, whose many sites make it difficult for them to meet their obligation of the 80/20 cost share with FCSAP.  The cost sharing formula between FCSAP and custodians is generous, however.  For projects over $10 million, the cost share rises to 90/10, and in exceptional circumstances, projects can be entirely funded by FCSAP.  Correspondingly, the financial challenges that a few custodians face in addressing contaminated sites are the result of competing priorities within departments, rather than insufficient FCSAP funding.  

The evaluation found that the time of the year when Custodians receive FCSAP funding (October/November) creates project delays for most Custodians.  Custodians can mitigate this problem somewhat by requesting multi-year funding.  Unfortunately, the timing of the funding is a result of the Government of Canada business cycle and beyond the control of the FCSAP Secretariat.

Attracting and retaining staff who are qualified to address contaminated sites is another challenge that Custodians and Expert Support departments face.  The FCSAP Secretariat has provided some program management funding to FCSAP participants; however, it is ultimately the responsibility of Custodians and Expert Support departments to ensure that they have adequate staff to address their contaminated sites.  In coming years, the relative priority that Custodians and Expert Support departments give to federal contaminated sites will determine the pace at which federal liability can be reduced.

Reporting

The structures of the FCSI and IDEA databases are appropriate and the range of information they capture is sufficient to measure the progress of the program.  There are two pieces of information not currently captured in FCSI that would be helpful to the program.  The first would be a field identifying a site as an FCSAP funded site, and the second would identify a site as requiring no further action.  

Recommendation 7: The Treasury Board Secretariat, Real Property and Materiel Policy Division, should consider adding the capability for FCSI to identify a site as an FCSAP funded site, and to identify a site requiring no further action.  

Although most Custodians are meeting their contaminated sites reporting responsibilities, a few are not.  A lack of timely and accurate reporting by six Custodians is making it difficult for the FCSAP Secretariat to fully measure, understand, and communicate the achievements of the FCSAP to date.  Despite the efforts of the FCSAP Secretariat to provide guidelines and training on reporting, these Custodians are not meeting their reporting responsibilities. The six Custodians in question have errors in their FCSI data that have yet to be reconciled (some date as far back as the 2005–2006 fiscal year) and/or have liability data that differ from what was reported in Public Accounts.  In an era when the federal government is placing increasing importance on accountability and results, it is important to ensure that Custodians’ contaminated sites data in the FCSI is complete and up-to-date.    

Recommendation 8: The FCSAP Secretariat, in cooperation with the Treasury Board Secretariat, Real Property and Materiel Policy Division, should consider various processes that could be implemented to ensure that Custodians improve the quality and timeliness of both IDEA and FCSI data reporting.  

Recommendation 9: The FCSAP Secretariat, in cooperation with the Treasury Board Secretariat, should consider providing additional guidance, training and costing model/templates to help Custodians provide more accurate and consistent liability estimates for projects.

4.2 Success

Progress Toward Immediate Outcomes

The evaluation found that FCSAP has demonstrated progress toward immediate outcomes.  A number of tools have been developed and made available to assist Custodians in managing projects and communicating with public stakeholders.  Health Canada tools for communicating with the public seem to be highly regarded by Custodians.  Awareness and use of these tools is generally good, but could be somewhat higher.

The number of FCSAP projects funded has increased from 281 in 2005–2006 to 500 in 2006–2007.  Many projects have completed assessment and are now undergoing remediation or risk management activities.  The number of remediation or risk management projects funded by FCSAP has increased from 89 to 210, and the number of sites that have reached Step 7 or higher has increased from 167 to 202 over that same time frame.  

Based on an analysis of FCSI data, the evaluation found that remediation activities reduced federal liability at FCSAP funded sites by $361.9 million from 2005–2006 to 2007–2008, according to FCSI.  Upward liability adjustments for existing sites and the discovery of new sites have combined to offset liability reductions associated with remediation activities somewhat.  The net result has been a $69.4 million reduction in federal liability associated with federal contaminated sites from March 2006 to March 2007.

The evaluation found that liability estimates are being adjusted over time as more is known about the extent of contamination of sites; however, there remains some degree of uncertainty by Custodians regarding the proper practice for estimating financial liability.  Liability experts consulted during the evaluation revealed that other approaches exist that may reduce the variability and uncertainty surrounding the estimation of liability.  At the moment, TBS is consulting with liability experts to determine if it would be advantageous for FCSAP to adopt certain elements of other approaches to contaminated site liability estimation.

Challenges to Achieving Outcomes

Although the work of consultants and the FCSAP Secretariat have contributed to FCSAP success, the sheer number of sites that must be assessed; a lack of qualified consultants, a lack of Custodial and Expert Support personnel; and the remote location of many sites limit the progress that FCSAP can achieve.  The evaluation concluded that the FCSAP objective of eliminating (or reducing as much as possible) the federal liability by 2020 should be re-examined given the number and complexity of sites that need to be addressed.  Currently, the program has not developed an estimate of the number of years required to address all sites.

Recommendation 10: The Treasury Board Secretariat, Real Property and Materiel Policy Division, should consider requesting that Custodians articulate in their contaminated sites management plans estimates of the number of sites they have remaining to assess, how many sites they anticipate assessing each year until all sites have been assessed, and their best estimates for when no further action will be required for these sites.  

Recommendation 11: The FCSAP Secretariat, in cooperation with the Treasury Board Secretariat, Real Property and Materiel Policy Division, should consider undertaking a comprehensive horizontal program capacity/resource assessment; where gaps emerge, determine risks and develop strategies to address them.


| ToC | Previous | Next |