Agency Training Plan (ATP) Resource Kit
Stage3 -Review
This stage...
...begins after the first training program is finished.
From that point begins an on-going process of monitoring the quality and effectiveness of training.
The information gained from this process is used to inform planning for the next ATP.
Evaluation / monitoring
On-Going Evaluation
There are many approaches to evaluation and monitoring. On a practical level, it is more efficient to have on-going evaluation in place so that you can refine or change arrangements if they are not meeting the desired outcomes.
If you wait until the end of the 12 month cycle to find out whether the training was effective, you will waste funds on training that could have been modified and improved.
Four Levels of Evaluation
The Kirkpatrick Model outlines four levels of evaluation:
to the program and/or the trainer
- Reaction to the program and/or the trainer
- Learning as a result of the training program
- Behaviour changed as a result of the program
- Results from the changes that are noticeable in the participant’s skills, knowledge and/or attitudes over the longer term; the benefits these have begun to have for clients, the organisation, colleagues, quality of service delivery etc.
Level 1 & 2 Evaluation
This can be achieved through an end-of-training evaluation form completed by the participants. The form can cover pre and post training program self-assessment of skills and knowledge, ratings from 1-5 of usefulness of content, effectiveness of training delivery, appropriateness of venue, quality of materials, level of understanding, further needs.
Level 3 Evaluation
This can be achieved by asking participants 3-6 months after the training program what they have changed in their skills, behaviour or attitudes as a result of the training; asking supervisors, colleagues and clients what they observe as positive changes in skills, behaviour and attitude.
Level 4 Evaluation
Evaluation at the results level asks that agencies consider the impact on the organisation as a result of training. To do this, a ‘snapshot’ needs to be taken which measures a work practice before and 6-12 months after the training, to identify resulting benefits.
Evaluating results in the Disability Services could also include analysing client outcomes for example, incidence of challenging behaviour, movement into less restrictive programs, increases in adaptive behaviour.
It is not realistic that all training be evaluated to the results level. Given that this level of evaluation can be resource intensive, agencies need to be strategic about what programs they choose to evaluate in this way.
Examples of pro-formas and procedures for evaluation strategies are in the Appendix 10 and Appendix 11.
Organisations typically focus on the first two levels of evaluation. However, there is great value in developing strategies for measuring behavioural change brought about by the transfer of learning into the workplace. This ensures that training does not happen in a vacuum, that agencies benefit and staff are supported in applying new learning in their job roles.