Kirkpatrick’s Model for Follow up Assessment

Evaluation

A strong initiative within the CyberWatch K12 Division results from a question, often omitted from professional development (PD) activities: have the PD courses actually changed teachers' behaviors in the classroom? CW K12 has implemented an evaluation strategy based on Kirkpatrick's Training model. Too often training has been offered to fill a need, without any measures of success. But are the courses actually presenting material that is helpful to the teachers, in a context that enables them to apply the lessons to the classroom? How much transfer of knowledge, skills, and attitudes occurs in the classroom after the PD training opportunities? Are teachers passing on what they have learned to the students? Is the knowledge gained still being used in a month, or a year? This is what the third level of Kirkpatrick's model describes and what our training model is trying to measure. Evaluation feedback is used to modify, and expand offerings, as well as lead to new courses and content.

Kirkpatrick's model has three major levels of evaluation. The first is the "smile sheet", where the student is asked whether they liked the course and the instructor, and the strengths and weaknesses of the course. The second level consists of a pre and post test which seeks to identify how well the material is learned. The third level is most often omitted in professional development and is CW K12's focus - has the material been applied? Evaluation begins with a preliminary skills assessment setting a baseline for the participants. As the courses progress, surveys are distributed which are targeted to how the information is being used in the classroom. After each course concludes, follow-up surveys are conducted, and individuals are randomly selected for personal interviews to see if and how they have continued to apply the skills several  months after course conclusion. This kind of intensive evaluation could not be done by the CyberWatch alone; it requires the partnership of the counties who not only have closer ties to the teachers, but also have the directive to evaluate how well CW's program is working and to show if the training is helping to meet district educational goals.

The lessons learned are applicable to all professional development activities. The CW K12 model provides varied approaches that demonstrate multiple routes to success, tailored to the needs of each institution. The two most important lessons to be gathered from this PD model are (1) design educational activities which are flexible, and (2) incorporate evaluation and improvement as an integral part of your program.

References

Kirkpatrick, D., (1998). Evaluating Training Programs: The Four Levels. San Francisco: Berrett-Koehler

Li, Y. & Achilles, C. (1999-2000) Four variables in teachers' integrating technolgoy behaviors - A case study of a school in a Michigan school district. Forum of Teacher Education Journal, 10 (1), 12-21.

Roblyer, M., & Edwards, J. (2000). Integrating Educational Technology into Teaching. Upper Saddle River, NJ: Merrill

Rodriguez, G., (with Knuth, R.) (2000). Providing professional development for effective technology use. Pathways to School Improvement [Online]. Available: http://www.ncrel.org/sdrs/areas/issues/methods/technlgy/te1000.htm
Smith, G. & Caris, M., (2001). Teaching college courses: online vs. face-to-face. T.H.E. Journal. 28 (9), 18-15

Wilson, S., and Berne, J. (2000). Teacher learning and the acquisition of professional knowledge: an examination of research on contemporary professional development. Review of Research in Education 24, 173-209