Skip booklet index and go to page content

Follow-up Audit of Finance and Corporate Branch's IT Controls over Financial Systems

Annex 1: Audit Criteria

Remedial actions in response to the recommendations of the Phase 2 Audit Readiness Assessment have been undertaken and have been effective in addressing the findings from that report.

Sub-criteria
Sub-criteria No.DescriptionOverall ConclusionContinuous Monitoring Required?Control Design sustainable?Additional Audit Work Required?Comments
1User accounts for contractors and temporary staff can be identifiedLow riskYesYesYes

While the January review indicated that little work had been done surrounding the policy and procedures for this control, a thorough document review and interviews during the testing phase indicated that, as of April 1, 2011, the basic tools available to make this control work have been in place, including fields for establishing employee types and for end-dating accounts for term, contract and casual employees.

The implementation of the control was announced Department wide in late March of 2011. During interviews, we were told that. [Text removed to protect the security of the system]

As the control was only put in place on April 1, 2011, [Text removed to protect the security of the system] there was insufficient new user activity to test, and as a result the audit team could not conclude on how well these controls are working. The auditors did find a number of [Text removed to protect the security of the system]. The audit team has recommended further audit follow-up work after the September 30 date to establish the effectiveness of the control as implemented.

Given that appropriate procedures and policy for the new controls appear to be in place and [Text removed to protect the security of the system], we feel it poses a low risk to the Department's ability to provide auditable financial statements.

2Controls related to identity management in PWGSC service provisions have been specified in the service level agreement SLAMetNoYesNoAs per a review of the current SLA with PWGSC (February 2011), Environment Canada now has the necessary read-only activity for performing monitoring and audit activity. 
3Ongoing segregation of duties has been assuredLow riskYesYesYesPolicy and procedural documentation are appropriate and supporting IT-based tools are available, but insufficient evidence of monitoring was available for testing, leaving the audit team unable to conclude on the operating effectiveness of the control. 
4Generic user accounts are identified and their activity is monitoredMetYesYesNo

The audit team found evidence that generic accounts were reviewed in March 2011, but there was no evidence of an ongoing periodic review of these accounts. There was evidence of follow-up action resulting from the March review, deactivating all but 11 accounts. Two of these 11 remaining generic accounts are required by Oracle to perform functions. Interviews with the responsible business managers revealed that they were aware that there are 9 remaining generic accounts; that the accounts are being used for an ongoing initiative; and that their use is being actively monitored.

The original finding did not require that there be no generic accounts. It only required that the business managers know the purpose of the accounts, determine who was monitoring the accounts and document people who had access to an account's password. Based on this understanding and the availability of compensating controls, we believe that this criterion has been met.

Activity related to accounts of this type should continue to be monitored closely.

The audit team responsible for testing the control found that this control was met with only minor issues. However, when we reviewed the original finding and recommendation, we found that this sub-criteria had been met. We do recommend that, when generic accounts are used, their purpose and use be well documented and activity carried out by the accounts be monitored closely.

5No active accounts are assigned to former employeesMedium RiskYesNoYes

Evidence was provided showing that a custom report has been created listing active users with last login date. A Last logon date of active users report, generated 30 March 2011, was also provided. [Text removed to protect the security of the system] Follow-up audit should be planned to confirm that these accounts are still required.

Policy and procedure documentation does not indicate the frequency of review of the Last logon date of active users report or the individual responsible for reviewing, analyzing and storing this report. Through inquiry it was confirmed that the Last logon date of active users report will be generated and stored for audit purposes on a go-forward basis.

A copy of the new draft Employee Separation Clearance Report and the new draft directive on employee separation clearance were provided. It was confirmed by inquiry that the Employee Separation Clearance Report must be signed off by Informatics and Finance to ensure that proper access to Merlin is deactivated.

Given that the control was only put in place in April 2011, evidence was insufficient to provide assurance that this control is effective.

[Text removed to protect the security of the system]

Given the compensating controls, we feel that the residual risk that arises due to our inability to test this control is medium. Policy and procedure documents could be enhanced in the area of reporting departures of account holders, including a description of the nature and timing of monitoring. [Text removed to protect the security of the system]

6Merlin application user accounts are reviewed periodicallyLow riskYesYesYes

Based on the newly implemented procedures, Environment Canada should have a good understanding of which Merlin application accounts are active, what responsibilities those accounts should be associated with, and the status of employment of the account holders. Once this baseline information is available, we consider that the described ongoing monitoring strategy will address the original March 2009 findings and associated recommendation.

While at the present time there is insufficient evidence to conclude that the process is operating effectively, we find that the policy and procedure environment and the associated IT controls provide sufficient compensatory control to result in a low level of residual risk. Audit follow-up work should be considered for the periods after September 2011 to establish how effective the monitoring process is.

7User authentication is monitored at the OS, database and application levels to identify suspicious activityMetYesYesYes

During document review, the audit team saw evidence that reports are available to monitor unsuccessful logins at both the application and the database levels. [Text removed to protect the security of the system]

Through interviews the audit team was informed that monthly reviews had not been performed in the past but that one was carried out on March 31, 2011. During interviews the audit team was informed that the plan is to perform [Text removed to protect the security of the system] reviews of unsuccessful logins at both the database and the application levels on a go-forward basis. [Text removed to protect the security of the system]

Policy and procedural documentation for application and database failed login monitoring is appropriate. While monitoring of failed login attempts has been found to be adequate, the monitoring should be documented in terms of what it is meant to accomplish, when it is conducted, by whom, what was found and what if any actions arose from the monitoring process.

8Activity by privileged Merlin users is monitoredMedium RiskYesNoYes

There are two methods of monitoring activity within Merlin.

The first method is governed by the Signon Audit Level, which monitors activity of users at one of four levels. The most detailed level is the form level. At this level, user activity is stored when the user logs in, opens or closes a form, or prints something.

The second method for monitoring activity is governed by the Audit Trail system profile option. This method of auditing records database changes at the record level, storing a before-and-after image for every change made. This method of creating an audit trail is very powerful, but it also consumes huge amounts of resources. It is set for a table or a group of tables.

[Text removed to protect the security of the system]

9User access control functions are monitored at the application levelLow riskYesNoYes

For this control to work, administrators need to know when someone has left the Department. The administrators then need to react by deactivating the associated Merlin account in a timely fashion. Administrators now have three ways of finding out when a user has left the Department.

First, they will soon have the revised Employee Separation Clearance procedure and directive. The revised procedure specifies more clearly the requirement to notify Finance and CIOB that an employee has left. The new procedure also requires the Branch to disable the employee's access to the corporate IT infrastructure and financial systems.

Second, administrators are maintaining a monthly snapshot of active users on the system. By comparing one month to the next, it will be possible to see which users have been added to the system, been deactivated on the system or had a change in access privileges during the month. This control will not help system mangers deactivate accounts, but it does provide a tool for monitoring how well the other parts of the control are working.

A third method provided by a new report is available that shows the last login date for each employee, and employees that have not logged in recently can be subjected to further scrutiny.

Taken together, these three strategies mitigate the likelihood that an account will remain active for a departed employee.

Finally, since access to Merlin is only available across the LAN if the user's active directory account has been disabled, the presence of an active Merlin user account is less of a risk ([Text removed to protect the security of the system]).

We have noted, however, that the tools for accomplishing this work are not optimized for use by the business area. This weakness may lead to a control that is unsustainable over time because it will take too many resources to perform. [Text removed to protect the security of the system]

Policy documentation can be enhanced for monitoring and reporting related to the [Text removed to protect the security of the system]. Iinsufficient evidence was available for testing this control and as a result we were unable to conclude on its operating effectiveness with an audit level of assurance. We do find, however, that the anticipated control to be provided by the new directive and forms [Text removed to protect the security of the system] will mitigate this risk to a low level by reducing the likelihood [Text removed to protect the security of the system]. The detective controls that are now available (monitoring the active user reports) will further mitigate this risk. Even so, we still find that these monitoring controls have not been optimized for this task, increasing the risk that monitoring will be unsustainable over time.

10Merlin application password controls are monitoredMetYesYesNo

On April 1, 2011, a new password control regime was implemented at Environment Canada. The audit team found that the new regime is an effective control of passwords for Merlin users and that it addresses all of the concerns raised in the March 2009 report finding and fulfills all of the recommendations from that report.

The audit team reviewed the password settings in the application configuration and found that the settings meet or exceed the settings recommended in the Phase 2 Audit Readiness Assessment.

Further, when the controls were tested in the presence of the audit team, they were found to work as established.

11User activity is logged at the application level to the form levelMetYesNoNoWe found through interviews and document review that the recommended level of access logging has been set in the system. We find that this sub-criterion has been met and that the control is adequately implemented.
12Database administrator's accountability is assured [Text removed to protect the security of the system]Low riskYesYesYes

By reviewing the roles assigned to the database administrator's individual accounts. [Text removed to protect the security of the system]

As was noted in the Phase 2 Audit Readiness Assessment report, this can pose a significant risk to the organization. [Text removed to protect the security of the system]

The audit team also noted that [Text removed to protect the security of the system].

We were told that the DBA team with access to Oracle Financials is relatively small, they work in close proximity, and the individual DBAs are cleared to the secret level. Finally, with the [Text removed to protect the security of the system]. These compensating controls mitigate somewhat the risks posed by the [Text removed to protect the security of the system].

We find that this control is ineffective, but that the compensating controls noted above do reduce the level of risk associated with the weakness to low.

13System database accounts are identified and have appropriate password controlsLow riskYesNoYes

The tools are now in place to make this control work, but there was insufficient evidence to conclude that the tools are being used to effectively mitigate the risks described in the Phase 2 report. [Text removed to protect the security of the system]

Further, the tools that have been proposed for this activity are not optimized to help system managers monitor this activity. This may make the monitoring process unsustainable over time.

Policy and procedural documentation does not indicate that passwords will be changed on a regular basis and insufficient evidence was available to test the control's effectiveness. As a result, the audit team was unable to conclude on the operating effectiveness of controls.

Given the weaknesses in the documentation [Text removed to protect the security of the system] which is contrary to best practices, we find that this control is still ineffective but poses only a low level of risk to the Department's ability to produce auditable financial statements.

14Merlin database user accounts are periodically reviewedMetYesNoNoThe audit team found through document review and interviews that this control is effective. We found that the tools are in place to allow for monitoring the creation/change/deactivation of Merlin database accounts. However, as with other monitoring controls in this system, we find that the tools that have been developed are not optimized for the job for which they are intended. This means that the monitoring controls may not be sustainable over time.
15Standard procedures have been established for creating new database accountsLow riskYesYesYesThe audit team found insufficient evidence to conclude that standard procedures or controls exist relating to requesting, approving, creating and segregating duties for new database accounts and access rights. We further found that the control that is in place is ineffective in that documentation made it appear that users could request enhanced access to the production platform under their own authority. Even though this turned out not to be the case, the lack of standards on how to record this type of activity leaves the Department at a low level of risk.
16Standard procedures have been established to ensure that database account privileges are changed when users change positions or responsibilitiesLow riskYesYesYesThe audit team found insufficient evidence to conclude that standard procedures or controls exist relating to requesting, approving, creating and segregating duties for new database accounts and access rights. We further found that the control that is in place is ineffective in that documentation made it appear that users could request enhanced access to the production platform under their own authority. Even though this turned out not to be the case, the lack of standards on how to record this type of activity leaves the Department at a low level of risk.
17Standard procedures have been established for [Text removed to protect the security of the system]Low riskYesYesYes

Policy and procedural documents do not indicate that monitoring will be conducted on a regular basis. [Text removed to protect the security of the system]

Given that the tools for monitoring this type of activity were only implemented in April of this year, there has been insufficient time to accumulate records to test. Given the inability to test this control and the weakness in the documentation, the audit team found this control to be ineffective [Text removed to protect the security of the system]. However, given compensating controls like the plan to monitor this activity in the future, we feel that the level of risk posed by the control weakness is low.

18Standard procedures have been established to ensure that program changes go through testing and quality assurance prior to being promoted to productionLow riskYesYesYes

Through document review it was found that the documentation for this process is out of date, still making reference to the Software Management Board, which was recently replaced by the Change Control Board (CCB).

Other documentation indicates that the CCB formally approves, rejects or defers change requests. The CCB is supposed to meet on a monthly basis. Per inquiry it was confirmed that the CCB is a new board that has not yet had a formal meeting. The meetings are to occur on a go-forward basis.

Through inquiry it was confirmed that a form is used to capture approval by all module leads before the ticket or change request is promoted to another environment. A signed copy of the form is kept by the manager and test script results are kept in a binder by the module lead. A sample form was provided.

The audit team had asked for a system-generated list of program changes so that that list could be used as a sample frame from which to select a sample for testing. As the list could not be generated, testing was not possible. In absence of the ability to produce such a list, some mechanism is required for generating a log of program changes by date that indicates who approved each promotion of code from one platform to another. Follow-up audit work should consider using the output from the ticketing system as a sample frame.

There is an opportunity for improving the documentation of program change procedures and the program changes themselves. While insufficient evidence was provided to allow us to conclude on the risk at an audit level of assurance, we find that this criteria was largely met with minor issues and that these issues pose a low level of risk to the Department’s objectives.

19The test environment is identical to the production environmentMetNoYesNo

It is not possible to have a completely identical test environment unless the test environment is maintained on an identical but physically separated network. The test environment must exist in a different Oracle instance which, by necessity, has a different identifier if it is installed in the same network environment. There are a number of other installation parameters that must be different between the two environments so that the programs from one environment do not affect the databases on the other environment.

The audit team responsible for testing the control found it to be ineffective, but given the need to have some differences between the environments, the documented method of cloning and refreshing the production environment for use in test, development and QA is more than adequate and provides reasonable assurance that code in one environment will act the same way as the same code in the other environment.

Policy and procedural documentation is acceptable and, based upon the method used for cloning the production environments, we find that this sub-criterion has been met.

20Change request forms for Merlin database and application are appropriately authorizedMetYesYesNoConfirmed through observation and inquiry that the change request template enforces a digital signature.
21Formal processes exist for monitoring changes made to the Oracle database and to the Merlin applicationMetYesYesNoIt was confirmed through observation and inquiry that the QA team will sign off on the release of change request tickets. 
22Post implementation review work by the QA team is documented and retainedMetYesYesNoIt was confirmed through observation and inquiry that the QA team will perform post-implementation review and retain evidence related to the review.
23Operating system user access is monitoredLow riskYesNoYes

The audit team responsible for the testing did not find sufficient documentation describing how operating system accounts are managed. Further, the audit team noted a few anomalies like evidence of a problem during the upgrade of the system from the UNIX platform to its current LINUX platform. It is possible that these problems would have been caught before they occurred if adequate standards had existed. The audit team found a few isolated cases that need follow-up via the monitoring process and in future audits.

We discovered by inquiry that Environment Canada employees only ever get Read/Execute access to file shares on the OS. They are never granted admin rights on the OS.

While insufficient information was available for comprehensive testing, the audit team has concluded that this control and accompanying documentation are inadequate. Given that the level of access granted to Environment Canada employees is restricted to file access in specified shared areas, the level of risk posed by this weakness is low [Text removed to protect the security of the system] will be required to ensure that this risk remains low.

24A list of people authorized to request that PWGSC create OS accounts is created and maintainedMetYesYesNoThrough inquiry and document review we have discovered that PWGSC has been provided with a list of Environment Canada staff who can authorize the creation, modification and/or removal of operating system accounts for Environment Canada employees. Further, through inquiry we have determined that a separate list exists at PWGSC that identifies which Environment Canada employees must be notified when an operating system account is added, changed or deactivated. We have concluded that these two controls, taken together, provide adequate control.
25Processes exists to ensure that user accounts for departing employees are deactivated in a timely fashionLow riskYesYesYes

The audit team responsible for testing this control found insufficient evidence to provide an audit level of assurance that this control is adequate. Coupled with a lack of current documentation surrounding the process, the audit team concluded that the control was ineffective.

Through document review and inquiry, we did discover that the anticipated directive (and supporting forms and procedures) on employee separation clearance is expected to provide IT staff with advance information about the departure of an employee. Part of the associated IT process is to ensure that all system accounts are deactivated for employees as they depart. This new directive is expected to take effect in July of 2011. Further, the new service level agreement with PWGSC gives Environment Canada managers read-only access to the operating system, enabling future monitoring and auditing of the process.

With these two improvements to the control environment expected in the near future, we conclude that this control weakness only poses a low level of risk to the Department. To maintain this low level of risk, follow-up audit work must be carried out to ensure that monitoring is taking place and that the new directive and associated procedures are approved and implemented.

Page 5
Date modified: