Formative Evaluation of the Federal Contaminated Sites Action Plan

Final Evaluation Report

| ToC | Previous | Next |


3.0 Evaluation Findings

This section provides the evaluation findings for issues of design and delivery as well as results.  A number of evaluation questions that are interrelated have been grouped together.

3.1 Design and Delivery

Q1: Is the FCSAP governance model comprehensive, clear and appropriate for a horizontal initiative?

Q 2: Are roles and responsibilities clear and appropriate for a horizontal initiative such as FCSAP? Who is accountable for the program? Is there an appropriate horizontal accountability framework?

Finding: The governance model is comprehensive, clear, and appropriate for a horizontal initiative.  Horizontal roles and responsibilities are clear and appropriate, and a horizontal accountability framework is in place.    

EC and TBS jointly administer the Federal Contaminated Sites Action Plan. Within EC, the FCSAP Secretariat provides program oversight and administers the non-financial aspects of the program. EC manages the project selection process, maintains a secure website, develops communication materials, and monitors and reports progress. TBS ensures the program’s adherence to Treasury Board policies on the management of federal real property, reviews the financial aspects of proposals, assesses custodians’ reallocation capacity, administers the fund, and advises the FCSAP Secretariat on the monitoring of government-wide progress in addressing federal contaminated sites funded under FCSAP.11

The FCSAP Steering Committee recommends strategic direction, approves the work plans of the secretariat and the Expert Support departments, guides the development of the strategic plan, approves funding options, and ratifies funding recommendations. The Steering Committee oversees program implementation and is responsible for setting project priorities, monitoring progress, and providing recommendations on the funding of sites under FCSAP.

The FCSAP governance structure is presented below. 

Exhibit 3.1 FCSAP Governance Structure

The exhibit presents the FCSAP Governance Structure, which includes the FCSAP ADM-level Steering Committee (co-chaired by Environment Canada and Treasury Board Secretariat), the FCSAP Secretariat, as well as the Contaminated Sites Management Working Group.

FCSAP documentation describes the FCSAP governance model in a comprehensive, clear and appropriate manner for a horizontal initiative.  The documents, and in particular, the FCSAP Results Based Management and Accountability Framework, contain detailed descriptions of program components such as the FCSAP Secretariat, Expert Support, FCS Steering Committee, CSMWG, and IRWGs; their responsibilities and accountabilities; and how each contributes to the achievement of the FCSAP’s objectives.  

Evaluators discussed the governance model, and program roles and responsibilities, with a wide range of Custodians, Expert Support, and FCSAP Secretariat representatives.  The majority of those interviewed believe that the roles and responsibilities of the FCSAP Secretariat, Expert Support, Custodians, and committees such as the FCS Steering Committee and CSMWG, are perceived to be clear and appropriate, and that accountability structures are adequate.  

Like any large interdepartmental program, FCSAP does have some challenges, such as achieving effective communication. A few Custodians complain that program information at headquarters does not always flow to the regions on a timely basis, and at the same time, ADMs and DGs can be unaware of what is happening in the regions.   

Custodians and Expert Support noted that the IRWGs provide an opportunity for Custodians to access advice from Expert Support.  A thematic case study on the role of the IRWGs conducted for the evaluation revealed that members would find the meetings more useful and attendance would improve if workshops and training sessions were offered.  In addition, a common frustration voiced by IRWG members was that issues raised at the IRWG meetings are either not passed along to the CSMWG or FCSAP Secretariat, or they are passed along, but no feedback is provided to the IRWG on the issues that were raised. 

Q 3: To what extent has FCSAP been implemented as originally planned?  If not implemented as planned, what was the rationale for different implementation?

Finding: In general, FCSAP has been implemented as originally planned, with two key exceptions: low ADM attendance at Steering Committee meetings and the absence of a Third Party Expert Panel.

The evaluation found that for the most part, there is consistency between the original plan, as described in program documentation, and what has actually been implemented.  Some program augmentations have occurred and are viewed as positive.  Key changes were recently made to the program to provide more flexibility to Custodians.  The first was the pre-approval of one-year assessment and remediation activities and the second was the decision to allow Custodians to internally reallocate FCSAP remediation and risk management/care and maintenance funds in-year between approved projects.  This change was made to allow for more flexibility in using funds and to allow transfers between activities and between projects. 

Since FCSAP began in 2005–2006, TBS has attempted to expand and strengthen requirements for three-year custodial management plans and annual reporting, and was part of the organizing committee for two national contaminated sites workshops hosted by the Real Property Institute of Canada.  These activities were not foreseen when FCSAP began.  

Two deviations from the original plan for FCSAP have occurred.  First, the FCS Steering Committee, which meets twice per year, is intended to provide strategic oversight and act as a challenge function to the decisions made by the FCSAP Secretariat and CSMWG.  The evaluation found that many ADMs do not attend these meetings, sending DGs and Directors in their place.   According to the ADMs and DGs interviewed, the reason for low ADM attendance is that meeting agendas, set by the FCSAP Secretariat, focus on technical rather than strategic issues.  Accordingly, many ADMs feel that the level of the discussions at these meetings is not well suited to their role(s). 

Second, according to program documentation, a Third Party Expert Panel was to have been created to provide an external perspective including independent examination, independent advice, and the provision of an extra level of discipline.  After operating for two years without the panel, it was decided that the establishment of a panel would be postponed pending recommendation from the program evaluation on the requirement for and value of the panel.  

Q 4: Are decision-making processes in place to allow for the highest areas of importance to be reflected in the allocation of resources (priorities)?

Q 5: Is risk appropriately addressed? 

Finding: Decision-making processes are in place, and in general, resources are allocated to the projects of highest priority.  Risk is addressed appropriately, but there are some weaknesses.  On occasion, inexperienced consultants conduct risk assessments that may lead to inappropriate remediation strategies.  In rare cases, external pressures can influence the prioritization of projects by Custodians.  Other related findings include: opportunities to risk manage rather than remediate sites are being passed over; long-term monitoring of high risk sites is underutilized; the benefits associated with project closure reports are not being realized because the reports are not mandatory; and the likelihood of a risk managed area or requirement being overlooked in future years is significant because restrictive covenants on land title identifying the risk-management requirements and/or areas are not required by FCSAP.  

A review of FCSAP documentation found that the program has well-documented procedures, standards for assessing risk, a risk ranking methodology, and other related guidance documents produced by experts in the field.  Remediation, risk management, and care and maintenance projects proposed for funding are ranked using the science-based FCSAAP ranking methodology and criteria approved by TB.  The methodology uses risk measurement tools developed by Expert Support departments in consultation with the CSMWG to evaluate the human health and ecological risks associated with a site.  Other factors taken into consideration for ranking projects include special considerations (impact of contamination on traditional lands and lifestyles; increase in financial liability over next five years if no action is taken; and potential legal liability or requirements).  

A review by evaluators of remediation projects submitted by DFO, DND, INAC NAO, and TC revealed that average scores for projects were 193 points (out of a maximum of 400) in the 2005–2006 TB submission and decreased to 168 points in the 2008–2009 TB submission.  There are some instances where remediation projects with very high scores were submitted in 2008/09; however, it is likely that several years of assessment were required before these sites could be submitted.  The downward trend in ranking scores suggests that the highest risk projects have been allocated program resources first.

Exhibit 3.2 Ranking Scores of Projects Submitted for Remediation in 2005/06 and 2008/09 TB Submissions

 

2005–2006 TB Submission 2008–2009 TB Submission

Dept

MIN

MAX

Average

MIN

MAX

Average

DFO

134

175

152

78

150

117

TC

178

219

197

138

284

193

DND

146

287

184

110

247

154

INAC 

97

361

204

117

266

181

Total

97

361

193

78

284

168

In general, the evaluation found that resource allocation and the manner in which risk is addressed are appropriate, but there is room for improvement.  First, a few senior and highly experienced industry representatives pointed out that how well risk is assessed depends on the skill level of the people who conduct the risk assessments, which they believe can vary considerably in some cases.  Although risk assessments conducted by inexperienced consultants may not impact whether or not a site gets funded (program funding is sufficient to ensure that all sites that meet the basic eligibility requirements are funded), it may impact the remediation strategy and financial liability estimates of sites.  For instance, a risk assessment that underestimates the actual contamination may lead to a remediation strategy that is not aggressive enough, and financial liability estimates that are too low.

Second, a few Custodians, Expert Support, and industry representatives stated that on occasion, external pressures influence Custodians in choosing the projects and sites to address with FCSAP funding.  It was reported that some sites are cleaned even though that might not be the best use of resources.    

Third, some industry and Expert Support representatives pointed out that Custodians tend to be risk averse and proceed with full remediation when Expert Support or consultants recommend a risk management approach.  For instance, in the case study of the Pointe Noire Interpretation and Observation Centre (DFO), the Custodian now believes, in retrospect, that it might have been more beneficial—financially and in terms of human health and environmental risk—to manage the risks rather than excavate the Point-Noire site.  External scientific experts echoed the views of Expert Support.  They suggest advancing the practice of risk management of sites and allowing flexibility to use regional or provincial guidelines for tolerable contaminant levels.  The FCSAP Secretariat and a consultant are currently developing a Decision-Making Framework that is slated for release in March 2009, which should guide Custodians in making these kinds of decisions.    

Fourth, a few Expert Support interviewees stated that long term or periodic monitoring after project completion should be pursued for highly contaminated sites, and/or those that have used an engineered approach such as a constructed landfill, dam, or dump, in case they fail to work as planned (e.g., melting permafrost can compromise the functioning of a frozen landfill).  

Fifth, a few Expert Support interviewees mentioned that an FCSAP project closure report should be required for all sites.  Currently the provinces of British Columbia, Ontario, and Quebec require, for provincial sites, project closure reports that summarize the original problem, the solution pursued, and any remaining risk or concern that should be noted.

Finally, an Expert Support interviewee pointed out that where some sites are being risk managed (e.g., capping ground with a half metre of soil rather than removing all contaminated soil), no formal mechanism exists to ensure risk management and monitoring activities are uninterrupted in future years.  Some provinces place a restrictive covenant or other instrument on land title identifying the risk-management requirements and/or areas.  This mechanism ensures that any new property owner is aware of the contaminated area and his/her responsibilities for risk managing and monitoring.  The Federal government has no such requirement.  The likelihood of a risk managed area or requirement being overlooked in future years is significant, especially as federal Custodians have a considerable number of pending retirements and associated contaminated sites knowledge may be lost. Although record keeping is in place for the program, evaluators do not consider record keeping to be sufficient to avoid this risk.

Q 6: To what extent are FCSAP activities and outputs appropriately linked to shared outcomes and the overall horizontal mandate?

Finding: In general, program activities and outputs link appropriately to shared outcomes and the overall mandate.  Custodial contaminated sites management plans, however, are not clear in demonstrating how actions of individual Custodians will contribute to the overall objectives of FCSAP.

The evaluation found a logical link between FCSAP activities, outputs and the outcomes desired by the program. The program’s logic model was recently updated in Spring 2007.  

FCSAP has greatly increased the financial resources at the disposal of Custodians to address contaminated sites.  The resulting increase in funding has not only accelerated the management of contaminated sites, but has also contributed to a number of secondary program objectives, such as skill development and employment opportunities for people living in Canada’s northern communities, technology development, and human resource development in the private sector.

Expert Support departments provide advice and develop tools to help Custodians complete more rigorous site assessments and make better-informed decisions concerning the relative benefits of remediation and risk management. Health Canada for instance, has developed tools and provided advice to assist Custodians in addressing the concerns of the public at sites in close proximity to communities.  

The evaluation found two weaknesses in the link between custodial activities and desired program outcomes.  First, custodial contaminated sites management plans are not clear in demonstrating how actions of individual Custodians will contribute to the overall objectives of FCSAP.  The Status Report of the Commissioner of the Environment and Sustainable Development to the House of Commons (March 2008), noted that except for DND, plans did not include a clear objective to contribute to the broader government objective of effectively eliminating the known liability related to contaminated sites by 2020.  TBS has since issued updated guidelines to Custodians on what to include in their next round of management plans.

Second, circumstances beyond the control of FCSAP could break the link between remediation and the reduction of federal liability.  It was pointed out by external scientific experts and private industry representatives that legal risks and potential impacts on the health care system are not reflected in liability estimates and do not reduce in step with remediation activities.  For example, people who develop health problems who live (or lived) near a site that was remediated and whose liability was reduced to zero, may nevertheless sue the federal government if they believe the contaminated site caused their illness. However, in examining the link between remediation and the reduction of federal liability, it is important to note that the items to be included in a liability are governed by accounting standards set by the Canadian Institute of Chartered Accountants (CICA) which do not allow the recognition of liability in the financial statements of an entity to be based purely on speculations.12 Therefore, it is not expected that Custodians account for these possibilities when estimating liability; however, it is an issue of which the Secretariat and FCS Steering Committee should be aware. 

Q 7: Are all components of the FCSAP appropriately resourced?

Finding: Funding to assess and remediate sites is adequate in most cases, with two exceptions:  DND’s assessment funding needs often exceed the amount that the program can provide to them, and due to its large number of sites, DFO struggles to meet its obligation of the 80/20 program cost share. 

Approximately 90% of annual FCSAP funding is directed to projects and the remaining 10% is allocated to expert support, management, and administration.  Based on interviews with Custodians, the evaluation found that funding for site assessment and remediation is adequate, with two exceptions: DND, a Custodian with large and complex sites to assess; and DFO, a Custodian with the largest number of sites receiving FCSAP funding.  DND representatives indicated that funding needs for assessment often exceed the amount that the program can provide.  DFO representatives consider that because of its large number of sites, their department struggles to meet its obligation of the 80/20 program cost share.  

Key informant interviews, case studies, and the document review revealed that Custodians and Expert Support have difficulty attracting and retaining staff to work on contaminated sites.  Human resource gaps and difficulties in retaining staff are especially acute in the regions.  A few Custodians regard FCSAP as a sun-setting program and choose to hire temporary staff, which makes it very difficult to attract highly skilled staff.  The Giant Mine case study highlights the challenges in addressing sites with limited human resources.  Even if the mine’s remediation plan was in full swing, there would be ongoing concerns regarding staffing.  A large part of the project team is either retiring or being reassigned (four out of seven people are leaving).  

Key informants, particularly Expert Support, pointed out the need for money to develop more refined tools.  Examples that were mentioned include terms of reference modules for site assessments, and soil screening criteria.

Expert Support representatives stated that resource redirections impact their ability to help Custodians adequately address risk.  For instance, one Expert Support interviewee stated that they used to conduct site visits, which added considerable value to the Ecological Risk Evaluation Level II scoring process (ERE2), but due to budget redirections within their unit, they now conduct the majority of ERE2s from their office.  However, resource re-directions within custodial and/or Expert Support departments are beyond the control of FCSAP.

Ensuring adequate human resources are in place to address contaminated sites is the responsibility of Custodians and Expert Support departments since FCSAP was intended to accelerate the contaminated sites activities that Custodians were already pursuing.  Recognizing the demands that FCSAP funding has placed on human resources within custodial groups and Expert Support departments, the FCSAP Secretariat has provided funding to address their challenges in managing the program (e.g., funding to hire staff).

Q 8: Is the funding process supportive of FCSAP objectives?

Finding: The receipt of FCSAP funding in October/November works against the program’s objectives.  FCSAP has made multi-year funding available to mitigate the problem.  

Custodians and industry representatives stated that the timing of the receipt of FCSAP funding in the fall works contrary to remediation objectives, considering that most work must take place in summer months, especially at northern sites.  A representative from an engineering firm who has worked on many contaminated sites over many years noted that not having FCSAP funding in the summer is a major problem for smaller departments that have limited funds to cash manage.  It has been the interviewee’s experience that the cash flow problem has completely stalled progress in some instances.  

The timing of this funding is a result of the Government of Canada business cycle and is unfortunately beyond the control of FCSAP.  The FCSAP funding cycle is also a result of the program requirement for annual project submissions.  This centralized approach may have been appropriate in the original program design (FCSAAP) because of the need to ensure that very limited resources were directed only to a few highest-priority sites, but it may no longer be necessary.

Custodians can alleviate this problem to some extent by requesting multi-year funding; however, funding for the first year of the project will not be received until the fall, which will delay the start of work until the following spring unless the Custodian is willing to cash manage activities in the interim.  

Q 9: Is the performance measurement sufficient?

Finding: Performance measurement is FCSAP’s weakest point.  Some Custodians are having a difficult time in providing the FCSAP Secretariat with up to date and complete data on their activities and accomplishments.  Correspondingly, annual reporting by the FCSAP Secretariat is behind schedule.

According to program documentation, Custodians are required to:

Based on a review of FCSI and IDEA, the evaluation found that the structures of the FCSI and IDEA databases are appropriate and the range of information they capture is generally sufficient to measure the progress of the program.  There are two pieces of information not captured in FCSI that would be helpful to the program.  First, there is no field identifying a site as an FCSAP funded site.  Second, unlike IDEA, which has a field that clearly states whether a project is completed or not, there is no field in FCSI that clearly identifies a site as requiring no further action.  At the moment, sites are coded based upon their NCS Classification, namely, “action required”, “action likely required”, “action may be required”, “insufficient data”, or “action not likely required”.  Custodians may close sites when no further action is required, but there can be a time lag between the point at which a site requires no further action and when it is closed.  

The FCSAP Secretariat has developed several guidance documents to assist Custodians in the area of performance monitoring, measurement, and reporting requirements for FCSAP. For example, available documents for this review include: Guidance for Completing the FCSAP Reporting Forms, Walkthrough for FCSAP Submission Process on IDEA, and Project Submission Procedures.  Despite these guidance documents, some Custodians continue to struggle with reporting. Based on interviews with key informants concerning the quality and currentness of the information, and a review of both databases, the evaluation found that the information in IDEA and FCSI is inconsistent, and some Custodians have found it challenging to maintain up-to-date information in FCSI.  Federal Custodians are required to report to the FCSI and certify annually the completeness and accuracy of their records in the FCSI, as stated in the mandatory TB Reporting Standard on Real Property.  At the moment, certification is pending for three Custodians because it has not yet been confirmed if the liability information reported in FCSI matches that reported in Public Accounts.  TBS has not accepted certification for another six Custodians because there are errors in their reported FCSI data.  Furthermore, at the time this report was prepared (FY 2008–2009), certain Custodians were still entering site data for FY 2005–2006. 

According to key informants, policy breaches occur when Custodians do not report their information in the required timeframes.  Because the data is not reported in a timely fashion, it is difficult to reconcile the liability information reported in FCSI against what has been reported in the Public Accounts of Canada.  The problem, it was explained, is that Custodians are not meeting reporting requirements because of tight deadlines coupled with confusion around liability estimates and public accounts reporting.  Key informants from the FCSAP Secretariat observed that there seems to be a disconnect between the project level (i.e. Custodian or environmental consultant) and national headquarters level in the manner in which information is interpreted and reported.  In response to this challenge, an FCSI interdepartmental working group was established to address issues and errors. 

The FCSAP Secretariat produces a FCSAP annual report based mostly on IDEA data. However, due to limited staff to devote to developing an annual report, and inaccurate, incomplete and untimely data supplied by Custodians in the early years of the program, the 2005–2006 and 2006–2007 FCSAP annual reports were only recently developed.  The evaluation found that the information in the annual report is useful in understanding the activities undertaken and progress achieved to date.

Q 10: To what extent has the FCSAP produced key outputs as planned? 

Finding: In general, key outputs are being produced as planned.  The delays in completing FCSAP annual reports and strategic plans stand out however.  Evaluators could not determine if the completion of site assessments is on pace, as it is not apparent that targets were set.

According to the FCSAP Logic Model, Environment Canada is responsible for producing a number of outputs, including procedures and tools; a risk ranking methodology; TB submissions, strategic plans and annual reports; technical training materials and workshops; and the IDEA database.  Based on key informant interviews and document review, the evaluation found that these outputs have been produced as planned with two exceptions.  Only one strategic plan has been developed for FCSAP (they were to be updated annually), and the FCSAP annual reports have not been developed and finalized as promptly as anticipated.

TBS has developed its outputs as planned, namely providing advice on funding eligibility and liability recording; and managing and improving FCSI, where required.

Expert Support departments have developed assessment protocols and ranking tools, and have assisted in developing risk scores for projects.  They have also developed communications materials (HC); provided guidance, training materials, and workshops; and reviewed and approved risk scores.  PWGSC has developed a number of project management tools and provides advice and services regarding the management of contaminated sites.  

Custodians have produced outputs such as site assessments, risk assessments, liability estimates, annual reports, contaminated sites management plans, and reports in IDEA and FCSI.  Despite the fact that some Custodians identified their planned site assessments in the contaminated sites management plans, evaluators could not determine whether site assessments are being conducted “as planned” because it is not apparent that targets at the program level were ever set.

Custodians made some suggestions for additional outputs, such as modifying the classification system and establishing guidelines for certain contaminants (e.g., zinc, nickel).  A CSMWG sub-committee (Aquatic Sites Working Group) was created in 2007 to develop guidance for classifying, assessing and managing federal aquatic sites, and a Request for Proposal for the development of the classification system for aquatic sites is under way.

Expert Support representatives suggested that some additional outputs be targeted to facilitate the achievement of horizontal outcomes, including stronger outputs in relation to site closure; a linking mechanism at a more senior level to ensure public involvement is in the assessment process; integrating precise engineering capabilities into the project assessment process; and, stronger focus on communication tools/public involvement.

3.2 Success

Q 11: To what extent has the FCSAP achieved immediate outcomes?

a) Increase in the number of Class 1 & 2 federal contaminated sites for which remediation/risk management plans are being developed or have been developed and/or implemented.

Finding: Remediation/risk management plans were developed for 59 projects in 2006–2007.  The number of sites undergoing remediation or risk management increased from 126 sites in 2005–2006 to 424 sites in 2006–2007.  

According to the IDEA database, remediation/risk management action plans were developed for 59 projects in 2006–2007 (similar data is not available for 2005–2006).  From the FCSAP Annual Reports (2005–2006 and 2006–2007), we can see that the number of projects undergoing remediation or risk management has increased.  In 2005–2006, 89 projects representing 126 sites were funded, and by 2006–2007, the number of projects had increased to 210 (424 sites).  Approximately one-third of these projects are DFO light stations.

Exhibit 3.3 - Number of Remediation/Risk Management Plans Developed and Projects Funded

 

2005–2006

2006–2007

Number of Remediation/Risk Management Action Plans Developed

N/A

59

Number of Remediation/Risk Management Projects (Sites) Funded 

89 (126)

210 (424)

Source: FCSAP Annual Reports, 2005–2006 and 2006–2007; and IDEA database

The 2007–2008 reporting data has not been finalized in IDEA at the moment.  

b) Greater reliability of FCSI and financial liability estimates.

Finding: Custodians remain uncertain regarding the proper practice for estimating financial liability, and there has been a tendency to underestimate liability to date.  

Custodians indicated some degree of uncertainty with the current practice for estimating financial liability. Some interviewees from Expert Support departments reported that the process for liability estimation is still being refined and suggest that the quality of estimation will improve as the refinement process proceeds.

Some respondents stated that it is difficult to really know what the remediation cost will be until remediation work is in progress, and the estimated value is likely to vary from the initial estimate.  These statements were supported by a review of FCSI data that revealed that more than half of the sites that first received FCSAAP funding in 2003–2004 and 2004–2005 have since had their liability adjusted.  The review also found that site liability has been adjusted up two to three times more often than it has been adjusted down, suggesting that there has been a tendency to underestimate liability to date.  

A liability expert who was interviewed indicated that the federal government uses a different approach to estimate liability, compared to private industry and provincial and municipal clients. The FCSAP approach is to assess multiple sites as one project. Clients usually only include one property (which may be quite large). In addition, the method of ranking sites is particular to the federal government, and the process for moving from identifying a site that may be contaminated to liability estimation moves more quickly for non-federal properties.

Another liability expert suggested that other methods could be applied to limit the variability in liability estimation. Custodians use standard Environmental Site Assessment (ESA) to determine liability, which can increase variability. Other jurisdictions (e.g., the United States) use more structured methods. For example, a liability expert mentioned that the University of Tennessee uses an approach that combines risk assessment with sampling regimes, and geographic information systems (GIS).

Despite these challenges, it appears that estimates are gradually becoming more reliable over time as the program matures and more site assessments are completed.  There are at least two sources of evidence to support this finding.  First, as sites continue through the assessment process, adjustments are made to liability, so, over time, the estimates become more reliable.  Second, most interviewees stated that they believe that the estimates are becoming more reliable.  

c) More accurate picture of federal contaminated sites, ranked according to human health and ecological risks.

Finding: The picture of federal contaminated sites is more accurate today than when FCSAP began.  

Through the identification of potentially contaminated sites by custodians and the funding of projects by FCSAP, the picture of federal contaminated sites is becoming clearer with regard to the number of sites, the extent of contamination, and the estimated federal liability. . For example, in 2006–2007, FCSAP funded 500 projects, up from 281 projects in 2005–2006.

Exhibit 3.4 - Number of FCSAP Projects Funded13

Projects

2005–2006

2006–2007

Assessment

183

280

Care and Maintenance

9

10

Remediation/Risk Mgmt

89

210

TOTAL

281

500

Source: FCSAP Annual Reports (2005–2006 and 2006–2007)

Site assessment in particular is key to clarifying the picture of contaminated sites in Canada.  According to the FCSAP Annual Report (2006–2007), 48% of sites assessed indicated the need for more investigation; 14% indicated that they require remediation or risk management activities; and 38% required no further action.

Custodian reporting in FCSI also revealed that more and more sites are reaching and completing remediation and/or undergoing monitoring, which also supports the finding that the picture of federal contaminated sites is becoming clearer.  As illustrated in Exhibit 3.5, the number of sites reported in FCSI to have completed Step 7 or higher continues to increase.  

Exhibit 3.5 - Highest Step Completed 

 

Number of Sites

Step Completed

Reporting at end of FY2005–2006

Reporting as of July 8, 2008

Step 7

95

105

Step 8

53

68

Step 9

18

26

Step 10

1

3

Source: FCSI

Because assessment and remediation can be a long and involved process, progression through the ten steps can take many years.  A review of FCSI data revealed that most sites move less than one step per year, and many have to move backward to do more assessment.  Despite the long and involved process of addressing contaminated sites, federal liability is gradually being reduced.  A review of FCSI data14 reveals that remediation expenditures for FCSAP sites that directly reduced federal liability were $78.1 million in 2005–2006, $148.3 million in 2006–2007, and $135.5 million in 2007–2008.  The discovery of new sites and upward adjustments to liability work against the above liability reductions.  The net result, according to the FCSAP Annual Report (2006–2007), is that federal liability for FCSAP funded sites decreased by $69.4 million from March 2006 to March 2007.

d) Increased availability and awareness of nationally consistent management and communications tools to address highest risk sites.

Finding: A number of tools have been developed and made available to assist Custodians in managing projects and communicating with public stakeholders.  Awareness and use of these tools is generally good, but could be higher.

Evaluators discussed management's availability and awareness as well as communication tools with the FCSAP Secretariat, Expert Support, Custodians, and private sector consultants.  From these discussions it became apparent that Health Canada’s tools and guidance to assist Custodians to manage communications with community stakeholders are generally highly regarded.  

PWGSC has developed many tools to assist Custodians in project management, such as Project Initiation, Procurement Planning, Risk Management, and Quality Planning. The evaluation found that not all Custodians use project management and communication tools.  A number of projects take place a great distance from communities, so tools to communicate with the public are sometimes not used.  A few Custodians are of the opinion that the effort associated with learning and using PWGSC project management tools is not justified if they only manage a few sites. At the other end of the spectrum, some Custodians with many and/or large sites developed their own project management tools before FCSAP began.

Q 12: What is the likelihood that FCSAP will be able to achieve the anticipated intermediate outcomes within the timeframe indicated?  What will be the anticipated challenges/barriers to achieving and/or demonstrating these intermediate outcomes? 

Q 13: At this stage, what external factors have contributed to, or detracted from, FCSAP’s level of success? 

Finding: The likelihood that the FCSAP objective of eliminating (or reducing as much as possible) the federal liability by 2020 is challenged by various factors, including the sheer number of sites that must be assessed; a lack of qualified consultants, lack of custodial and Expert Support staff; and the remote location of many sites.

The evaluation found that while some Custodians appear confident that they can remediate all their sites by 2020, others are not.  Correspondingly, the program may need to be extended beyond 2020. 

The evaluation identified factors that have contributed to the success of FCSAP to date.  Key informants reported that external consultants contribute to FCSAP success by coordinating workshops and providing training.  Other positive forces include the work of the FCSAP Secretariat and other levels of government.  For example, provincial governments reportedly support divestiture activities and establish provincial guidelines for environmental assessment.  

A number of internal factors detract from the results that FCSAP can achieve. Expert Support representatives pointed out that a key barrier to progress is the insensitivity of the NCS site scoring approach.  For instance, sites can be very different in terms of the severity of contamination and risk but receive the same NCS score. The concern of many Expert Support representatives is that the sites being remediated are not necessarily the most critical ones.  These concerns are mitigated to some extent due to the fact that sites must meet basic eligibility criteria, and, to date, all sites determined to be NCS Class 1 or 2 have received funding.  

A lack of human resources within the government is a key factor limiting the progress of the program, according to Custodians and Expert Support.  It is reportedly challenging to fill staff positions even when the resources are available. 

DFO Expert Support pointed to lack of communication and integration of information across custodial sites, the need for stronger relationships between regions and national headquarters, and the need for better buy-in/cooperation from Custodians, as internal factors limiting the progress that FCSAP can achieve.  In their 2006–2007 Expert Support Activities Report, Health Canada noted that Custodians were experiencing difficulty with the completion of the Human Health Preliminary Quantitative Risk Assessment (PQRA), and required more training, as “there were still a few [that] were very poorly completed PQRAs that required considerable input and effort by HC to be able to score and rank the site.”

In the online consultation with industry, firms indicated there were a number of challenges in making substantial progress toward outcomes desired by FCSAP.15 For instance, the discovery of new sites will work against decreasing the number of Class 1 and Class 2 sites.  Other challenges mentioned by firms included: the perception that procurement processes are slow; involving multiple stakeholders tends to slow down progress; the age of initial site data can be quite old and require updating; some large sites may never be fully remediated and must undergo care and maintenance; and the cost of reassessing sites to keep up with the new and ever - hanging Health Canada and Canadian Council of Ministers of the Environment guidelines.  

A number of factors outside of the control of FCSAP detract from the results it can achieve.  For example, the INAC Congress of Aboriginal Peoples Review 2002–2006 indicated that INAC “faces a number of challenges that will impact its ability to meet the objectives, including: 1) Climate changes are making the window of opportunity to put in winter roads very small; 2) An increase in commodity prices is resulting in higher demand for resources, both human and equipment, and in turn is reducing supply and increasing costs; and 3) Time and money must be consumed on agreement with local governments and the negotiation of land claims.” 

Several case studies such as FOX-M DEW Line, demonstrated that northern site locations are challenging due to poor weather, long winters, and infrequent sea and plane lifts.  In the case of Giant Mine, significant community pressure has stalled a remediation plan for the mine. 

Other detracting factors beyond the control of the program include insufficient human resource capacity in firms that are qualified to conduct FCSAP work.  A lack of qualified practitioners with the required expertise was reported to be a challenge for Custodians looking for assistance on contaminated sites projects.

Q 14: Have there been any unanticipated results, either positive or negative, that can be attributed to the program? If so, how were they addressed? If so, to what extent are these results related to the FCSAP being a horizontal initiative?

Finding: The discovery of contamination on sites in close proximity to FCSAP sites has been an unanticipated result.   Progress has also been made toward achieving secondary objectives such as employment opportunities for residents in northern communities, and developing and adopting innovative approaches to remediation and risk management.

The evaluation found that one unanticipated program result has been the discovery of sources of contamination that Custodians did not intend to target.  For example, contaminants have been found on properties that are not Crown responsibility, which are in close proximity to FCSAP sites. This is positive because these sources might not have otherwise been discovered.

The evaluation identified a number of secondary FCSAP objectives.  Some positive economic opportunities and capacity building were identified for Aboriginal communities in the North.  The cleanup of the DEW Line sites in particular has been an excellent skills training ground for the Inuit community.  It should be noted that the high level of aboriginal employment on FCSAP projects in certain regions is the result of requirements of various land claim agreements, and hence the benefits that flow from this employment are not solely attributable to FCSAP.   

The skills that Aboriginal people have learned on DEW Line cleanup are transferable to other professions such as mining, according to Custodians and industry representatives who have worked on these contaminated sites.  Furthermore, training programs have been developed for workers in northern communities (e.g., heavy equipment operators, etc.), which have created more qualified workers.  Now that this benefit has been recognized, a few respondents suggested better planning and coordination to train local people in remote communities. 

Other positive impacts include the sharing of information that occurred through bringing together different Custodian and Expert Support departments.  Synergies and coordinated efforts have occurred through national workshops, as networking and information sharing occur between departments and regions. Stakeholders have also coordinated the sharing of equipment and human resources in remote regions, which is very effective in reducing costs. In addition, awareness has been raised to the point where Custodians have become proactive in preventing new site contamination, and an increase in ecological integrity is being recognized in departmental activities outside of FCSAP.  

Another key impact of the FCSAP has been to encourage the provinces and municipal governments to become more active on the contaminated sites front.  In some cases, these governments have implemented more stringent standards than the federal government.

Some innovative technologies and approaches to remediation have been developed and implemented on FCSAP sites.  For instance, a land farming approach is in place at FOX-M DEW Line where the soil covering the landfill is turned once per week to break down hydrocarbons faster.  Bacteria are added to the soil to help break hydrocarbons into carbon dioxide and water.  In addition, berms were constructed around the landfill to freeze contaminated soil in place.  

According to some industry representatives, the program has served as a great training ground for consultants and engineers.  This assertion was confirmed in program documentation which stated that the demand for skills and services generated by FCSAP has helped create new jobs in the environmental industry and projected that employment in this sector will at least double over the life of the program.


| ToC | Previous | Next |


11 FCSAP Annual Report, 2006–2007 (2008), p.5.

12 CICA Handbook: PS 3200.08

13 It should be noted that DFO projects often consist of several properties, which contain many sites.  For example, for the assessment in 2006–2007, FCSAP funded 36 DFO projects which comprised 650 sites.

14 The review of the FCSI database was conducted on July 8, 2008.

15 It should be noted that the challenges described in this section are not unique to FCSAP and are faced by other entities attempting to address contaminated sites.