How to perform a recovery simulation test for IT departments

This is a great way to test how your IT department will fare in the case of a serious business disruption. It runs through the process of moving various tapes, materials, personnel, etc., to a disaster recovery site, as well as setup of the equipment and communications necessary for alternate processing by the IT recovery team.

Participants

  • Business recovery coordinator
  • IT recovery team personnel, as appropriate
  • Other personnel as determined necessary

Procedures

  • Business recovery coordinator
  • Distribute the entire plan, including blank copies of all applicable exhibits, to each participant
  • Monitor the test and note changes, discussion points and observations for later discussion with the team leaders
  • Reassemble the teams following completion of the test to discuss general observations, ideas, suggestions and team-specific changes or suggestions relating to the plan
  • Meet with the team leader and alternate individually to discuss specific suggestions and changes that will enhance the plan and the future testing environment
  • Evaluate the group’s performance in simulating the disaster scenario and identifying enhancements to the plan
  • Update the plan based on the feedback from the team leader/alternate
  • Maintain a file of maintenance forms that will provide evidence that the review and test have been performed
  • Retain a representative sample of materials, including test results, in a file for review by regulators, third-party auditors or internal auditors

IT recovery team leader

  • Develop an estimate of costs that may be incurred relating to the test and obtain management approval prior to proceeding
  • Meet with all participants to explain the purpose and scope of the test
  • Explain to the participants that backup materials and supplies may be retrieved only from the off-site storage location for this test
  • Develop a written schedule describing major activities, time frames and responsible persons for each major activity within the test
  • Track the number of hours used for testing and be cognizant of limitations and potential additional costs
  • Track expenses relating to the test and review alternatives following the test that could potentially lower expenditures
  • Lead the post-exercise discussion with team customers to review the results of the exercise and improvements to be made in the plan or exercise
  • Manage the development and publishing of changes to the IT recovery team plan
  • Prepare the post-exercise report for management and the business recovery coordinator, noting:
    • Exercise objectives and participants
    • Date exercise was performed
    • Activities performed
    • Degree to which the objectives were achieved
    • Statements about identified revisions to the plan or that no revisions were required
    • Recommendations to further improve preparedness
    • Comments and future exercise recommendations

    IT recovery team

    • Move the necessary materials (e.g., magnetic tapes, reference manuals and supplies) from the off-site storage facility to the alternate processing site
    • Discuss the following with the team leader before starting the simulation:
      • Notification responsibilities by various team customers
      • Coordination responsibilities to and from the vendors
      • Critical forms and supplies that may be needed
      • Backup and off-site storage of production files
      • Rotation of IT staff
      • Use the IT recovery team plan to restore the computer and communications environment
      • Consider changes in the security of operating and application systems that may be necessary to operate in an emergency mode
      • Verify the usability of the restored environment by accessing data and simulating processing
      • Check the configuration of all network equipment and test communication circuits
      • Note changes and enhancements that may be appropriate, and discuss these with the business recovery coordinator and team leader following the test
      • Discuss staffing requirements as described in the plan
      • Supplement minimum staff with temporary resources, if needed
      • Prepare and assist in plan updates or enhancements