June 2008
Acronyms used in the report
| AEB | Audit and Evaluation Branch |
| CESD | Commissioner of the Environment and Sustainable Development |
| CGA | Certified General Accountant |
| CGAP | Certified Government Auditing Professional |
| CIA | Certified Internal Auditor |
| DAEC | Departmental Audit and Evaluation Committee |
| DEC | Departmental Evaluation Committee |
| EAAC | External Audit Advisory Committee |
| EC | Environment Canada |
| EMC | Executive Management Committee |
| HR | Human Resources |
| IIA | Institute of Internal Auditors |
| MAF | Management Accountability Framework |
| OAG | Office of the Auditor General |
| PM | Person month |
| TBS | Treasury Board Secretariat |
| TRS | Time Recording System |
| SPC | Strategic Policy and Coordination |
Environment Canada's Audit and Evaluation Branch (AEB) provides the Deputy Minister and senior management with objective, independent and evidence-based information, assurance and advice on management practices, controls and information, and on the performance of programs, policies and initiatives to enhance results-oriented and accountable management, based on “best practices” and professional standards (of the Treasury Board and the Institute of Internal Auditors).
Each fiscal year, the Branch prepares a departmental annual plan and an annual report on its activities as well as a status report on internal audit engagements and evaluation projects conducted during that year.[1]
The purpose of this document is to report to the External Audit Advisory Committee (EAAC) and the Departmental Evaluation Committee (DEC) on accomplishments achieved against commitments and priorities for 2007–08, as set out in the Audit and Evaluation Plan 2007–10, and important lessons learned throughout the year.
Of the 13 internal audit projects planned for 2007–08, five internal audit reports were completed and approved during the past fiscal year. More particularly, three internal audit reports were approved by the Departmental Audit and Evaluation Committee (DAEC) and two reports were recommended by EAAC to the Deputy Minister for approval. Six additional projects were carried out in 2007–08 and will be finalized for consideration by the External Audit Advisory Committee in April 2008. Another project was initiated and one was postponed to 2008–09.
Internal audit also provided active leadership in the implementation of the TBS Policy on Internal Audit across government by participating in Chief Audit Executive meetings and serving on the Senior Advisory Committee on Internal Audit Capacity, the Audit Committee Advisory Group, the Steering Committee on Community Development, the Model Organization Working Group, the Fundamental Controls Working Group, the Audit Software Working Group and the Funding/Business Case Working Group.
Table 1 provides the status of the internal audits included in the Audit and Evaluation Plan for 2007–08
The evaluation division originally planned to conduct 16 evaluation projects in 2007–08. One additional project was added to the list, bringing the total number of projects to 17.
Of that number, seven evaluations projects were completed and approved during the past fiscal year. One evaluation plan carried out in 2007–08 will be presented to DEC for information in April 2008. Three additional evaluation projects carried out in 2007-08 will be presented to DEC in April 2008 and two in June 2008. Three other projects were carried over into 2008–09 with another one postponed to that year.
Evaluation also provided leadership through membership on the TBS Committees on evaluation guidelines, performance measurement and evaluation.
Table 2 provides the status of the evaluation projects included in the Audit and Evaluation Plan for 2007–08.
A key accomplishment of the Strategic Planning and Coordination (SPC) division during 2007–08 was the establishment of the EAAC, the development of the operational framework and the provision of secretariat support for both EAAC and DEC.
Fourteen briefing sessions were provided to departmental senior management on the changes in the AEB governance structure.
The Branch also provided a liaison, coordination and support role for EC’s involvement in one chapter of the Auditor General (AG) and in 12 chapters of the Commissioner of the Environment and Sustainable Development (CESD) reports during 2007–08. The Department was also included in one audit conducted by the Public Service Commission of Canada. Table 3 provides a complete list of these audits.
During the last fiscal year, an annual update on 18 audit recommendations made between 2001–02 and 2006–07 was completed and provided to the Office of the Auditor General. The Minister of the Environment also received 50 environmental petitions from the CESD, and SPC facilitated the process and assisted departmental managers in responding to these environmental petitions.
In addition, the Branch refined its planning process by introducing a more rigorous approach that included a risk assessment that linked the risk areas to the Management and Accountability Framework (MAF) elements to develop the departmental Audit and Evaluation Plan for 2008–11.
During 2007–08, the Audit and Evaluation Branch undertook a number of activities to strengthen the internal audit, evaluation and strategic planning and coordination functions. This section summarizes some of the key accomplishments.
In 2007–08, the key Branch priority areas were:
The following provides a summary of progress made against these priorities.
In 2007–08, the Treasury Board approved the creation of a three-member, independent External Audit Advisory Committee (EAAC) whose fundamental role is to ensure that the Deputy Minister has independent objective advice, guidance and assurance on the adequacy of the Department’s control and accountability processes. They reinforce the independence of internal audit and, at the same time, challenge the function to achieve high levels of performance and professionalism.
The EAAC received training and orientation sessions not only from the Department but also from Treasury Board to help them fulfil their mandate.
Charters for both the External Audit Advisory Committee and the Internal Audit function have been developed and approved by the Deputy Minister. A three-year forward agenda has been developed for the EAAC. EAAC conducted three meetings and recommended the approval of two internal audit reports. EAAC declined to recommend the approval of management responses to recommendations in five chapters of the CESD report and returned to the department for further improvement of the responses prior to their submission to the Deputy Minister for approval.
During 2007–08, the Internal Audit unit began the planning work required to provide the first annual holistic opinion to the Deputy Minister on departmental controls, governance and risk management in spring 2009. More particularly, the Branch conducted a gap analysis of fundamental controls during the last fiscal year and is planning to carry out data gathering of fundamental controls in 2008–09.
The Branch also received additional TBS temporary funding of $377,500 in 2007–08 to implement the requirements of the Policy on Internal Audit.
In 2007–08, the Department established a new Departmental Evaluation Committee, chaired by the Deputy Minister, to provide oversight of the evaluation function.
The departmental Evaluation Policy was updated to reflect the roles and responsibilities of the Departmental Evaluation Committee. The Policy was presented to management and approved by the Deputy Minister in July 2007. The Branch also started some early planning for the updated TBSEvaluation Policy which is expected to become effective October 1, 2008. The revised policy will require evaluation of 100% of all direct program spending over a five-year cycle, including grants and contributions coverage.
In addition, the evaluation processes were updated to ensure early involvement in the review process for Memoranda to Cabinet and in the development of Treasury Board submissions for new and existing programs, policies and initiatives to ensure that the objectives and intended results are clearly defined, realistic and measurable.
An internal communication strategy for disseminating lessons learned from evaluation studies was developed and approved by DEC in March 2008.
During the last fiscal year, the Branch established and staffed an EX position to head the Strategic Planning and Coordination (SPC) division. The SPC division is responsible for developing the risk-based audit and evaluation plans and corresponding reports, and quality assurance to ensure that the Departmental Audit and Evaluation plans, reports and activities contribute to the Department’s and the government’s priorities for the use of audit and evaluation as keys to strengthening public service governance, accountability and transparency. SPC is also responsible to ensure that secretariat support is provided to the EAAC and DEC.
Internal capacity was consolidated in the area of financial auditing and certification. There are now four employees of the internal audit group who have a professional designation in accounting or certification in internal audit, i.e., two Certified Internal Auditors (CIA), two Certified General Accountants (CGA), and one Certified Internal Auditor/Certified Government Auditing Professional (CIA/CGAP). The Chief Audit Executive and one employee are also currently completing their CIA designation, and one employee is completing his Certified Information Systems Auditor designation.
As part of the AEB Quality Assurance Program, the Branch conducted an internal assessment of its compliance with internal audit and evaluation standards. The assessment demonstrated that the Branch is generally in compliance with the standards, with the exception of three areas of non-compliance.
The primary area of non-compliance with the Institute of Internal Auditors (IIA) standards is the external quality assessment, required every five years, which should have been conducted before January 2007. The external assessment originally planned for 2007–08 was postponed to 2008–09, allowing the Branch to be part of the pilot-projects for Practice Inspection that will be led by TBS next fiscal year. The other area of non-compliance with the IIA standards is the disclosure of non-compliance that must be reported to senior management. Measures are being taken to report to EAAC at the June 2008 meeting. The Branch is required to be fully compliant with Treasury Board policy, directives and guidelines on internal audit by April 1, 2009.
The area of non-compliance with the TB evaluation standards relates to competencies.
The TBS Centre for Excellence in Evaluation and the Canadian Evaluation Society have started work on evaluation-specific competencies. These will guide the development of departmental evaluation competency models. This process is also tied to the new TB Evaluation Policy currently being drafted. The resulting evaluation competency model will be reviewed against the departmental competency model. The departmental model addresses career planning, learning objectives and methodologies.
The draft TBS assessment of EC’s performance against the Management Accountability Framework for 2007 provides an “Acceptable” rating for the internal audit and evaluation functions. The Risk Based Audit Plan was one area in which EC’s internal audit was rated with “Opportunity for Improvement”. More particularly, the Audit Plan should have included a more defined and risk-ranked audit universe, clearly identify linkages with the Department’s Corporate Risk Profile, with risk ranking the selected audit projects. The Plan should also have provided a rationale for constraints or adequacy of resources estimates and for carry-over audits. The 2008–09 Annual Audit Plan has been improved to include a rationale for carry-over audits and a statement of adequacy of resources. The Branch will also be using the draft TBS Guideline on Risk Based Audit Plan to improve its audit universe and overall process for the development of the 2009–10 plan.
During the past year, the Branch has made progress towards implementing the strategies identified in the HR Plan for 2007–08. For example, the Branch provided coaching sessions to help employees determine their career objectives and the steps to achieve those objectives. Each employee developed a personal learning and development plan to acquire and maintain the knowledge, skills and competencies related to their level and functions, departmental business priorities and career aspirations.
In addition, the Branch conducted outreach activities by identifying Canadian universities offering related programs to increase visibility and attract people into the audit and evaluation field. It also increased the use of student programs by doubling the number of students hired last year in evaluation.
Figure 1 offers a comparison between the number of person months planned versus actual person months spent on projects and is based on information extracted from the Branch’s Time Recording Systems (TRS).
In 2007–08, the Internal Audit division had planned a total of 70 person months (PM) to complete 13 projects. The TRS information revealed that 59 PM were applied to audit projects in 2007–08, with a variance of 11 PM. Person month utilization was lower than expected for the following reasons:
The above reduction of 11 PM was counterbalanced with more PM than originally planned coded against extended sick leave, staff departure and professional training.
The Evaluation division had planned a total of 60 PM to complete 16 projects and used 63 PM. The variance of 3 PM is due to one new project added and completed in 2007–08, and one new project initiated.
The increase of one person month in Strategic Planning and Coordination is due mainly to more time spent on environmental petitions and the OAG audits.
Figure 1: Project-Related Person Months Utilization by Function
Figure 2 provides the people time spent on various activities and is also based on information extracted from the TRS. Overall, 72% of employee time was dedicated to audit (28%), evaluation (29%), strategic planning and coordination (12%), and support provided to EAAC (3%) during 2007–08. Approximately 28% was spent on other activities such as training, meetings, consultations, staffing, TRS and a number of administrative activities.[2]
Figure 2: Time Spent on Various Activities
The total resource requirement for 2007–08 was forecasted to be 3.3 million dollars. That planned budget was reduced to 2.8 million in August 2007 and included additional temporary funding of $377,000 from Treasury Board for the implementation of the Policy on Internal Audit.
The Branch expenditures for last year amounted to approximately 2.7 million dollars. Resources spent were lower than anticipated due to factors such as external funding for specific evaluations that did not flow through AEB. Figure 3 provides a breakdown of Branch expenditures for 2007–08 by function, including those related to management activities.
Figure 3: Actual Expenditures by Function [3]
A number of important lessons learned have surfaced as a result of the audits and evaluations conducted throughout the year. These lessons learned are presented as either being relevant to the overall Department or to the Audit and Evaluation Branch.
Lessons learned relevant to the overall Department
Early and ongoing senior management engagement is extremely valuable in successfully executing AEB projects. From a practical standpoint, the involvement of senior managers promotes buy-in to the evaluation process, which in turn facilitates access to more precise and exhaustive information, and allows for the early identification of risks and strategies for their mitigation. Overall benefits to the Department are also enhanced, as early involvement increases the relevance of evaluation findings, improves managers’ understanding of evaluation results, and allows for more timely action in response to evaluation findings.
Difficulty tracking resources makes it difficult for programs to tie resources to results. In particular, value for money, cost-effectiveness and management of finances cannot be assessed when comprehensive or reliable financial information is not tracked and/or not accessible. This can also negatively affect the proper functioning and effectiveness of programs, as managers may lack information which is crucial for proper program delivery.
Within the current results management structure where program accountabilities are often shared, incomplete tracking and allocation of funds may also lead to confusion related to program roles and responsibilities. Ultimately, accurate and complete tracking of allocations and expenditures is a necessary element of program accountability. AEB has already alerted senior management to these issues and proposed an approach to more clearly define and explore key problem areas and potential solutions.
Performance measurement continues to be poorly implemented throughout the Department. Consistently tracked performance data, including baseline data, is an important means of demonstrating the achievement of outputs and outcomes. Stronger performance data would contribute to more effective program management and would better support decision making, departmental reporting, and strategic reviews. AEB continues to engage the Corporate Management Directorate (CMD) in discussions related to the need to track performance information and the appropriate roles for their respective groups. As a follow-up to the recent Management Accountability Framework (MAF) Assessment exercise with Treasury Board Secretariat, CMD is currently developing a strategy to improve the departmental performance framework. As well, CMD is aware of the need to explore the question of whether and how to track the development and implementation of Results-based Management Accountability Frameworks (RMAFs) throughout the department.
Lessons learned relevant to the Audit and Evaluation Branch
The Audit and Evaluation Branch remains committed to conducting the highest quality of audit engagements and evaluation projects to support the improvement of management practices in the Department and is committed to continually improving and strengthening its functions to respond to changing demands and requirements.
For fiscal year 2008–09, the Audit and Evaluation Branch will focus its resources and efforts on the following priorities [4] :
Table 1 – Status of Internal Audit Projects Planned for 2007–08
| Project Name | Status/Comments | |
| 1 | Contingency Plan for Meteorological Service of Canada (MSC) Weather Prediction Program | Approved by DAEC in April 2007 |
| 2 | Travel Expenses and Compliance with Disclosure Policy | Approved by DAEC in April 2007 |
| 3 | Hospitality Expenses and Compliance with Disclosure Policy | Approved by DAEC in April 2007 |
| 4 | Disclosure of Reclassification of Positions (Part of the audit of mandatory disclosures presented in the 2007–08 AE Plan) | Report presented to EAAC in January 2008 and approved by the Deputy Minister in February 2008 |
| 5 | Disclosure of Contracts over 10K (Part of the audit of mandatory disclosures presented in the 2007–08 AE Plan) | Report recommended by EAAC in January 2008 and approved by the Deputy Minister in February 2008 |
| 6 | Canadian Wildlife Service (CWS) Control Self-Assessment | Report will be presented to EAAC in April 2008 for information. Project completed. |
| 7 | Decision Support Systems | Report will be presented to EAAC in October 2008. |
| 8 | Delegation of Financial Authority | Report will be presented to EAAC in April 2008. |
| 9 | Information Technology Security | Report will be presented to EAAC in April 2008. |
| 10 | Technology Infrastructure Directorate | Postponed to 2008–09 following senior management decision. |
| 11 | Gap Analysis on Fundamental Controls | Report will be presented to EAAC in April 2008 for information. Note: The project Data Gathering on Fundamental Controls originally planned for 2007–08 was modified to a two-phase approach project, i.e., Part I - conduct a Gap Analysis of Fundamental Control which was carried out in 2007–08 and Part II-Data Gathering on Fundamental Controls to be carried out during 2008–09. |
| 12 | Employment Equity | Report will be presented to EAAC in April 2008. |
| 13 | Financial Audit Plan | Project current under way and will be presented to EAAC in June 2008. |
Table 2 – Status of Evaluation Projects Planned for 2007–08
| Project Name | Status/Comments | |
| 1 | Commission for Environmental Cooperation (CEC) | Approved by DAEC in April 2007. |
| 2 | Georgia Basin Action Plan | Approved by DAEC in August 2007. |
| 3 | Canadian Regulatory System for Biotechnology (interdepartmental, led by Health Canada) | Approved by DEC in November 2007 |
| 4 | Canadian Shellfish Sanitation Program (interdepartmental with Canadian Food Inspection Agency and Fisheries and Oceans Canada) | Approved by DEC in November 2007 |
| 5 | Environmental Emergencies Program - including Public Security and Anti-Terrorism (PSAT) | Approved by DEC in March 2008 |
| 6 | Oceans Action Plan (interdepartmental with Fisheries and Oceans Canada) | DFO’s evaluation report presented for information to DEC in March 2008 |
| 7 | Border Air Quality Strategy: Air Quality Forecasting Component (New evaluation added during 2007–08) | Approved by DEC in March 2008 |
| 8 | Clean Air Agenda Evaluation Plan | Evaluation Plan will be presented to DEC for information in April 2008 |
| 9 | Toronto Waterfront Revitalization Initiative Contribution Agreement | Evaluation report will be presented to DEC in April 2008 without the final management response. Will return to DEC for approval of the management response in June 2008 |
| 10 | Transboundary Movement Program - under the Public Security and Anti-Terrorism (PSAT) funds | Evaluation report will be presented to DEC in April 2008 without the final management response. Will return to DEC for approval of the management response in June 2008 |
| 11 | Meteorological Services of Canada Transition | Evaluation report will be presented to DEC in April 2008 without the final management response. Will return to DEC for approval of the management response in June 2008 |
| 12 | Smog-Causing Emission Regulations in the Transportation Sector | Carried out in 2007–08 and will be presented to DEC for approval in June 2008 |
| 13 | Weather Predictions Evaluation Plan | Carried out in 2007–08 and will be presented to DEC for information in June 2008 |
| 14 | National Agri-Environmental Standards Initiative (NAESI) (Interdepartmental with Agriculture and Agri-Food Canada) | Evaluation plan completed in 2007–08 Evaluation carried over into 2008–09 |
| 15 | Federal Contaminated Sites Action Plan (interdepartmental – led by Environment Canada) | Carried over into 2008–09. |
| 16 | Chemicals Management Evaluation Plan (interdepartmental with Health Canada) | Decision was taken with Health Canada to postpone to 2008–09 |
| 17 | Strategic Environmental Assessment (Interdepartmental – led by the Canadian Environmental Assessment Agency) | Carried over into 2008–09. |
Table 3 – External Audits Involving Environment Canada in 2007–08
Office of the Auditor General (OAG) Reports The Auditor General tabled two reports to Parliament in 2007–08: one in May 2007 and one in October 2007. Environment Canada was included in the following chapter of the October 2007 report:
|
Commissioner of the Environment and Sustainable Development (CESD) Report The Commissioner tabled two reports to Parliament in 2007–08: one in October 2007 and one in March 2008. Environment Canada was included in 12 chapters: October 2007
March 2008
|
Public Service Commission of Canada (PSC) Environment Canada was included in one audit completed by the Public Service Commission of Canada in October 2007.
|
In 2007–08, the Minister of the Environment received 50 environmental petitions of which two were directed to the Parks Canada Agency (PCA) for response and two were received for information only. Of the 46 petitions remaining, 28 received responses within the 120-day legislated timeframe, 14 received responses late and 4 responses are under development. It should be noted that the Commissioner of the Environment and Sustainable Development reports annually on responses to petitions received between July 1 and June 30.
In addition to the above-noted audits, Environment Canada completed an annual update to the OAG on 18 recommendations made to the Department between 2001–02 and 2006–07.
[1] All reports are posted on Environment Canada’s web site at www.ec.gc.ca/ae-ve/default.asp?lang=En&n=3036EE9A-1.
[2] Miscellaneous category includes activities related to e-mails, supplies, information technology, weekly project updates, work-related reading, and student supervision. The other activities category includes things such as moving and the implementation of new policies.
[3] Includes salary and O&M expenditures.
[4] The Departmental Audit and Evaluation Plan for 2008–11 can be found at: http://ec.gc.ca/doc/ae-ve/rb-ae_644/toc_eng.htm.