6: Evaluation

Performance measurement and evaluation are complementary. While performance measurement is managed internally, ongoing and focuses on results of certain aspects of performance, evaluation is a snapshot in time conducted by an external organization. In short, ongoing performance measurement information is used in periodic evaluation to provide a more in-depth and independent assessment of the outcomes achieved.

Evaluation relies on sound performance measurement as a reliable and credible source of evidence to demonstrate progress toward intended program outcomes. Using information from performance measures and other evidence, evaluations can be helpful in identifying opportunities to improve the way in which future sustainable development activities are designed and delivered.  This, in turn, will also help the organization enhance how intended outcomes are achieved. In addition to design and delivery, evaluation can also be used to improve efficiency, economy, and provide oversight.

The FSDS and outcomes from the logic model will also be evaluated, reviewed and commented on at various stages.

6.1 Ongoing Formal Evaluations

Environment Canada’s Evaluation Division will conduct an evaluation when the FSDS is more mature and able to demonstrate progress towards achieving its intended outcomes, as identified in the logic model.  This evaluation will focus on Environment Canada’s role and will validate results achieved.  The results will be evaluated to ensure that they continue to align with the public’s needs and interests.

The FSDS will also be reviewed as part of EC’s evaluation of Environment Canada’s Sustainability and Reporting Indicators Program, scheduled for fiscal year 2012-13. The evaluation will be conducted using the new Treasury Board Policy on Evaluation (2009) and will focus on three primary issue areas: 1) relevance, 2) effectiveness, and 3) efficiency and economy. 

An internal audit of the FSDS is included in Environment Canada’s departmental risk-based audit plan, as required by the Treasury Board Policy on Internal Audit and the Directive on Chief Audit Executives, Internal Audit Plans, and Support to the Comptroller General.

The Commissioner of the Environment and Sustainable Developmentconducts performance audits and is responsible for assessing whether federal government departments are meeting their sustainable development objectives”.  As such, the CESD will be auditing the FSDS and departmental strategies and will report the results to Parliament.

6.2 Ongoing Informal Evaluations--Review and Comment

While the formal performance evaluation for the FSDS is outlined above, there is some scope for informal evaluation, in the form of review and comment, that is outlined in the Act and discussed in the Governance structure section. In order to ensure that the processes, tools and strategies used to develop FSDSs are sound, Environment Canada has established a comprehensive framework to check on key elements of the FSDS by:

  • Identifying opportunities for future iterations of the FSDS (for example, reviewing targets against SMART criteria – Specific; Measurable; Achievable; Relevant; Time-bound – to identify opportunities for improvement);
  • Determining needed improvements based on Progress Reports;
  • Identifying best practices and lessons learned;
  • Refining guidance to departments; and
  • Refining approaches used to develop, implement and evaluate the FSDS.

To that end, the FSDS will be evaluated at every stage of its development using the “Plan, Do, Check, Improve” model. For example, it is reviewed at the draft stage of the FSDS’s development as required by the Act [FSDA 9(3) (4)]. Comments can be provided by:

  • The Commissioner of the Environment and Sustainable Development,
  • The Sustainable Development Advisory Council,
  • Appropriate committees of each House of Parliament, and
  • Stakeholders and Canadians

Feedback from the above-mentioned is taken into account when drafting of the final FSDS.