Model for assessing the effectiveness of training by D. Kirkpatrick. Modern problems of science and education Donald Kirkpatrick's model for assessing personnel training

When is training effective?

Employees HR-departments and managers of most companies no longer need to explain the importance and significance staff training.

Difficulties now arise at the stage of assessing the real effectiveness of corporate business training.

The main goal of business training is to improve the organization’s business performance by improving the skills, abilities and behavior of its employees.

In addition, if the employer has a personnel training program, it is important factors when a candidate is looking for a new job. So if a candidate is going to improve his skills in the field of selling services, for example, is it better to do this at the expense of the employer?
But how can HR managers determine whether existing training and professional development are effective. Do they really help achieve the organization's goals? Donald Kirkpatrick back in the early 60s of the last century presented a four-level model for assessing the quality of educational programs, trainings, and seminars.

On the first level the emotional reaction of participants is measured business training for the training program.

It would seem that evaluating the training program at this level is not particularly important, since a positive reaction from the training participants will not yet guarantee the successful development of new skills and abilities.

But a negative reaction from participants can minimize staff motivation for development.

Kirkpatrick raises the importance of assessing this level, since further work on improving training programs becomes almost meaningless if it has not been possible to interest employees at this stage.

The consequence of this will be to complicate the assessment of the results of a business coach’s work, and at subsequent levels will require significantly greater HR efforts and resources.

At the first level, it will be enough to give questionnaires to the participants business training at the end of training, then collected and processed.

On the second level Kirkpatrick's model evaluates the results of the training completed by participants.

The customer of the training, HR, needs to understand to what extent The training participants managed to master exactly those skills and abilities for which this course was conducted and the training program was prepared.

To assess this level, you can use specially designed tests and tasks to determine the degree of development of these same skills, it is better to first digitize them (for example, on a scale from 1 to 10).

Then compare the results of the participants before the start of the training and after its completion according to the selected scale.

It was before the training: +2, and after it it became +5.

well, that means HR had something to fight for, and the “sales person” actually got something!

You should also remember about the method of simple observation of the behavior of participants during tasks and business games by the trainer.

This observation is then described in a report. brief characteristics participants.

But, an employee’s new knowledge has virtually no value for the company without the ability and desire to use it. But this is sometimes much more difficult.

Why, because the knowledge hidden in the head of a sales manager in itself does not affect the effectiveness of his sales.

Therefore, it is important that “awareness” occurs, that is, not only the information received directly from the trainer is learned, but also one’s own experience is gained upon completion of the training.

This problem can only be resolved third level Kirkpatrick models.

Quite it is possible to involve clients in the assessment training effectiveness.

For example, when evaluating sales training, you can use client questionnaires or analyze changes in the nature of entries in the book of complaints and suggestions in the event of undergoing training aimed at increasing customer-oriented services.

On the fourth level Kirkpatrick's model evaluates the impact of training on an organization's business results. This is a calculation level that requires preliminary calculations before training, such as calculating the return on sales for the entire sales department (salesmen - salesmen). And then the same calculation using the same methodology, but after the training program.

This stage and method is important for top management, since it is at this level that the final assessment of the effectiveness of investments in training activities is made.

Changes in the company's performance are measured both by qualitative indicators (changes in the company's image, brand awareness, improvement of the psychological climate, etc.) and quantitative ones (a N% decrease in staff turnover, an increase in sales volume by P%, etc.). Assessment at this level is the most complex and expensive.

The Kirkpatrick model has long been used by many HR managers to evaluate and improve training and professional development programs.

It allows the interests of three key stakeholder groups to be taken into account. The model helps top managers of companies make decisions: “Is the training worth the money and resources invested in it? Or should they be used for other purposes next time?” The employee will be able to find out how the training will affect their performance or have an impact on their career. Business trainer will be able to evaluate the real effectiveness of their training and work by expressing the result in the customer’s profit.

In 1954, one of the most famous models was created, suitable for both evaluation of training effectiveness, and for assessing the effectiveness of the learning process at all. Its author was an honorary professor at the University of Wisconsin (USA) Donald Kirkpatrick. He proposed dividing the learning process into four levels, which are subsequently assessed. In this way, in his opinion, it will be possible to ensure that the new skills acquired during the training process are applied directly in the workplace and, therefore, to achieve maximum results from this training.

Let's look at what these levels are and what needs to be assessed at each of them.

FIRST LEVEL – “REACTION”

At this stage, it is important to find out how those participating in the training react directly to the training itself - whether they like it, and what they will use the acquired skills and knowledge for.

To evaluate the effectiveness of the first level, tools such as response sheets (comment questionnaires, smile sheets, reaction questionnaires), interviews, and focus groups can be used.

Reaction sheets

The main task is to compose them in such a way as to get the maximum useful information. It is recommended to create from 8 to 15 questions for training participants, which should be both open-ended and with scales (for example, a “five-point scale” or “ten-point scale”) necessary for the most accurate assessment of the opinions of all training participants. Be sure to leave a space for comments next to each question. It is advisable to place questions on the response sheets that allow you to obtain information such as: whether the trainer and the methods he uses are effective, whether and to what extent the course objectives correspond to the goals of the activity, whether the material provided was accessible, how the program can be improved, etc.

It is advisable that the questionnaires be anonymous. It is believed that in this case the answers will be more frank.

Interview

An interview is conducted after training with each of the course participants in the same way as in the case of questionnaires, to obtain feedback.

Focus groups

Focus groups can be used in the case of pilot projects. It is recommended to collect the opinions of participants for the first time a week after the end of the project, the second time - 90 days after the project.

The result of the first level assessment can be expressed in answers to the following questions:

  • Who should this training be for?
  • Which methods will be more productive?
  • When should the training take place?
  • What did the participant get from the training?
  • How will he apply the skills and knowledge he has acquired in his workplace?

Thus, how participants respond to a course is how it is assessed. The reaction of training participants, according to D. Kirkpatrick, is a fairly important criterion for the success and effectiveness of the entire training course.

SECOND LEVEL – “TRAINING”

This level involves assessing the extent to which participants have learned new information whether they have formed the necessary relationships, whether and to what extent their knowledge and attitudes have changed at the end of the learning process.

To quantify progress in learning, specially designed questionnaires, assignments and tests are used.

The tools necessary to assess the second level of the model are: a test of knowledge of the material studied, a skill check sheet, drawing up an action plan, training other employees.

Skill Check Sheet

The skill check sheet is filled out by the observer, who, during a conversation with the course participant, determines his ability to use the skills acquired during training. In the first column of the sheet, the skill is assessed, in the second, comments are entered. Skills are assessed as follows:

1 - incorrect use of the skill;

2 - correct use of the skill;

3 - expert use of the skill.

Drawing up an action plan

To draw up an action plan, training participants are asked to answer the following questions:

  • What actions are you going to take?
  • What would you like to learn how to do?
  • What could stop you from achieving success?
  • How will you overcome the obstacles that arise?
  • What support will be needed for you to take action?
  • What results do you expect?

Another assessment tool is training other employees.

It can be organized in the form of conducting various presentations for colleagues from other departments, assistance in training and adaptation of new employees, or training a group in pairs.

According to D. Kirkpatrick, if the learning process was effective, then visible changes should occur, that is, knowledge will improve, skills will improve and attitudes will change.

THIRD LEVEL - “BEHAVIOR”

In this case, it is necessary to determine whether the behavior of its participants has changed as a result of the training, whether they apply any of the acquired skills and knowledge in their workplace.

The assessment tools used for the third level are as follows:

  • behavior checklist;
  • behavior review;
  • review of employee performance in the workplace;
  • review of action plans;
  • action learning;
  • focus groups.

The following questions can be used in the review test: whether new knowledge was gained; how much behavior at work has improved as a result of training; How long did it take to apply the training in your work? if it still hasn’t worked out, then why, etc.

D. Kirkpatrick draws attention to the fact that if changes in the behavior of the participants did not occur after the training, this does not mean at all that the training was not effective. There are cases when they were not created the necessary conditions, and, therefore, even with a positive reaction to the training, the behavior of the participants did not change. Therefore, when assessing, it is very important to check for the following conditions:

  • whether the participants have a desire to change their behavior;
  • do participants understand what and how to do;
  • whether an appropriate socio-psychological climate has been created in the workplace;
  • Is there any encouragement for training participants for changing their behavior?

FOURTH LEVEL – “RESULTS”

Assessing the extent to which pre-planned results have been achieved is the main goal of this level, i.e. This grade economic efficiency training. It is also important to note whether the changes in the behavior of the training participants had any positive impact on the organization itself.

Assessing the fourth level of the model is quite difficult, since here it is necessary to determine how the organization’s business indicators have changed. That is, you need to choose the right one indicators, which will be measured both before and after training.

D. Kirkpatrick notes that the results will include changes that occur as a result of participants completing the training (for example, increased sales, improved quality, increased productivity, etc.).

Assessment should be carried out before, during and after the learning process, and also after some time, when the results become more noticeable.

In the 1970s, D. Kirkpatrick’s model for assessing the effectiveness of training was already quite well known and accepted as a standard for assessing direct professional training.

Long years the author of the model conducted various studies, developing new tools and assessment methods, which were mainly related to levels 1 and 2. And only in 2005 they were offered the tools necessary to evaluate the 3rd level of the model. The task of ensuring the most effective application of knowledge and skills acquired through training in real activities is still quite relevant for various training organizations.

In 2006, Kirkpatrick released the third edition of his book “Evaluating Training Programs,” which focused on Level 4 of the model, and also significantly expanded the scope of the model itself. The author proposed that his model be used not only to evaluate the effectiveness of training programs, but also for the process associated with change management, and so that the value of business training could be demonstrated.

The algorithm for working with the model itself has also changed. In D. Kirkpatrick’s book, it is proposed to carry out assessments starting from level 4 and ending with level 1. Thus, in his opinion, specialists responsible for training will pay due attention not only to the learning process itself, but also to supporting the desired behavior and achieving planned business results .

According to the authors, in order to obtain guaranteed results, before starting training, it is necessary to determine the expected results, key indicators and the methods by which measurement and evaluation will be carried out.

FIFTH LEVEL - ROI

Also known Model level 5, developed Jack Phillips, thanks to which it should become clear whether it was worth investing in training. Its indicator is ROI(“return on investment”) allows you to convert the results of the fourth level assessment into a material equivalent, then the resulting amount of profit is compared with the costs of the training course. The ROI ratio shows percentage profit from the training course to the amount of costs for it.

Thus, the Jack Phillips model provides an opportunity to justify the cost of training, evaluate the training course as a business tool and shows the direct dependence of the increase in company productivity on the training of its employees, i.e. do training cost effective.

Despite the fact that Donald Kirkpatrick's model is over 50 years old, there are still many managers and training specialists who believe that once a learner has acquired any knowledge, he is already an expert and can easily apply it. But, as practice and the results of assessing the effectiveness of training according to D. Kirkpatrick’s model show, this is far from the case. The use of this model will allow us to improve in the future learning programs, as well as increase the efficiency of employee training.

The article is the property of a professional portal for trainers. When reprinting an article, a link to the site is required.

We recommend unique coaching techniques for the best exercises for training:

  • I'm great at it!

    Active game exercise, expanding the training participants’ self-image, increasing self-esteem and self-confidence and opening up new perspectives. The exercise reveals creative potential training participants, sets up and motivates the group for further work.
    Exercise “I can do it very well!” perfect for trainings personal growth and confidence. It can be successfully combined with the objectives of team-building trainings, and can be made very indicative of goal setting training. In addition, the exercise is indispensable for training in starting your own business and employment training.
    Volume of the training manual for the exercise: 8 pages.
    Bonuses! The technique contains 5 different variations exercises depending on the goals of the trainer and the characteristics of the group!

  • Exercise for training "Guess the hero"

    An interesting and revealing exercise for sales training, negotiations, communications, which in practice shows the effectiveness of using open and alternative questions when identifying customer needs. For this task, the “Guess the Hero” training exercise is unlikely to have worthy competitors. The manual contains 4 different variations of the exercise! Do you want toparticipants clearly saw the effectiveness open questions compared to closed ones? Or to realize what questions they most often use in practice? To practice asking effective questions? Everything is possible!

    Exclusive exercise manual developed by professional trainers especially for the portal Trenerskaya.ru. and contains unique recommendations,expert tips that reveal all the secrets of a successful exercise. You won't find this anywhere else!

    Volume of the training manual for the exercise: 9 pages.
    Bonuses! 4 different exercise variations and a detailed theory block!

  • Director's presentation


    A universal and effective exercise    , which can significantly expand the knowledge of training participants about self-presentation, assess their ability to make a first impression, work with confidence, verbal and non-verbal methods of communication, and at the same time speed up the process of assigning roles, put the group into a “working state” and bring it to the stage of active work.

    The exercise is suitable for all trainings that cover the topic of communications and presentations. “With a bang” goes through confidence and leadership training. It may be useful and effective in management training.
    The exercise can energize the group and motivate them to work hard.
    The training exercise was recommended by professional trainer D. Shvetsov, author of the book “Strengthening Personality.”

The game is worth the candle. How to evaluate the effectiveness of business training? Makota Elena Mikhailovna

Chapter 1 The Classic Model of Donald Kirkpatrick

Classic Donald Kirkpatrick model

Four levels of training effectiveness assessment

Most the right way to manage as much as possible - to hit the ground running. Time is money! If you haven't figured it out yet, my goal is to save as many of them as possible for you. So let's get to work immediately!

It is quite obvious that you can always measure this or that indicator in several ways. But is there one among them, the standard one? The one that guarantees you the most accuracy?

Not at all - no one can give you such a guarantee. This becomes especially clear when it comes to imprecise values: you can only calculate them approximately. Try to convince yourself that a 100-gram piece of cheese from the store contains exactly 28 grams of protein and not a gram less. Do you really believe this?

But - even adjusted for inaccuracies and errors - the basis of each assessment system is still a certain classical methodology or theory, recognized by the majority of specialists in a particular field.

System-forming theory. Fundamental. Verified. The one that allows us to speak with a high degree of confidence about the truth of the indicators obtained with its help. And just imagine - such a model also exists for assessing the effectiveness of training! Moreover, in this book we will analyze it in the most detail. But will she answer all our questions?

First, let's go visit the classics. Just as Philip Kotler became a marketing icon, Donald Kirkpatrick has achieved similar status in the American HR management. His four-level model for assessing the effectiveness of training was published in 1959 - already in those days, Americans approached the slogan “personnel decide everything” more scrupulously than planners Soviet Union. Damn prescient!

Look around and you will understand that true knowledge is not amenable to time. More than 50 years have passed, transnational corporations rule the roost, freelancers are increasingly replacing employees, and the Kirkpatrick model still remains one of the basic ones when assessing the effectiveness of staff training. What's the salt?

There is no secret here, but there is a competent grouping of information. In fact, in his model, Kirk-Patrick ordered and divided the entire sum of the effects of the training into four separate levels. The assessments of each of them, put together, give us information about the need and effectiveness of the training provided.

As I mentioned earlier, this model has four levels. Here they are:

Level 1:"Reaction". The main question is: “What is the employee’s reaction to training?”

Level 2:"Education". The main question: “What exactly did the employee learn during the training”?

Level 3:"Behavior". It analyzes how the skills and knowledge acquired during the training are applied in work. After this, you will be able to answer the main question of this level: “How has the professional behavior of students changed?”

Level 4:"Results". The company's performance indicators are directly assessed - how much have they changed after staff training? The main question: “What did the company get from the training?”

However, the study of the issue was not limited to the four levels mentioned. In 1991, the Kirk-Patrick model was expanded by Jack Phillips, who included a fifth and final level—return on investment (ROI).

This was a conscious attempt to translate the measurement of training effectiveness into a material plane. Whether Phillips succeeded or not, we will find out a little later. In the meantime, let's turn again to the classics, which, it turns out, are far from sinless.

Yes Yes! It turns out that the Kirkpatrick model has its own serious drawback - it evaluates the effectiveness of the training after it has been carried out. And not immediately: collecting data at all four levels can take several months. Do you and your business have time to wait?

Many managers answer this question negatively, preferring to limit themselves to assessment only at the first level - this saves time and financial costs. But will such an assessment be objective, will it show specific learning results, will it give necessary information for managers? Of course not - and you will receive evidence of this throughout this book.

 From the book Ears Waving a Donkey [Modern social programming. 1st edition] author Matveychev Oleg Anatolievich

From the book Management author Dorofeeva L I

5. Classical school in management Management knowledge appeared long before our era and long before management became an independent scientific discipline and profession. Management was recognized as an independent field of activity only in the 20th century.

From the book Management: lecture notes author Dorofeeva L I

1. Classical school in management Management knowledge appeared long before our era and long before management became an independent scientific discipline and profession. Management was recognized as an independent field of activity only in the twentieth century.

From the book Competence in Modern Society by Raven John

The Competence Model and the Psychological Model of Ability Before moving on to further discussion of this model of competence, let us consider its fundamental differences from the multifactorial model of ability that is so popular in the psychological literature.

From the book Marketing 3.0: from products to consumers and further to the human soul author Kotler Philip

Chapter 2 Marketing Model 3.0 A Brief Historical Review of the Last 60 Years of Marketing For the past six decades, marketing has been one of the most exciting topics in business. In short, marketing covers three main disciplines: product management,

From book Active sales 3.1: Beginning author Rysev Nikolay Yurievich

Classic technique - “Assume” The technique is that you invite the client to imagine that the problem he is talking about has already been solved, as if it doesn’t exist. P: And if there was still space on the counter, then what kind of sausage would you have you ordered? And here it comes

From the book Psychology as Business. How can a psychologist promote himself? author Chernikov Yuri Nikolaevich

Model 1.0 and model 2.0 – find a thousand differences Model 1.0 is an older generation model, a design from yesterday. In this case, we graduate from a university, then improve our qualifications to the level of a trainer-psychotherapist, psychologist-consultant, then become

From the book 100% brand. How to sell happiness author Lyaporov Vladimir Nikolaevich

From the book Management Styles - Effective and Ineffective author Adizes Yitzhak Calderon

From the book Functional Management. How to create order out of chaos, overcome uncertainty and achieve success author Ryatov Kadirbay

Chapter 1 Functional model of a person 1.1. Man and the world around. Actions as a source of knowledge For thousands of years, people have been accumulating knowledge about the world around them, honing the most important and necessary skills for life. Gained knowledge and experience

From the book How to become rich author McIver Meredith

Part 1 Donald J. Trump School of Business and Management In The Art of the Deal, I mentioned my nemesis, New York Military Academy mentor Theodore Dobias (to my left). To my right is Major General John

From the book How to Save on Marketing and Not Lose It author Monin Anton Alekseevich

Part 2 Your Apprenticeship Time (Donald's Advice for Advancement) Take Control of Your Interview Over the years, I've had some interesting experiences with job interviews. Good example– Norma Foerderer. After the first

From the book Great Team. What You Need to Know, Do, and Say to Build a Great Team by Miller Douglas

From the book Management Practice by human resourses author Armstrong Michael

From the book Universum. General theory management author Maslikov Vladislav Ivanovich

From the author's book

5.1. Classical and universal schemes The idea of ​​the unity of algorithms inherent in a wide variety of objects, phenomena and processes related to psychology, sociology, philosophy, information systems and many other areas of knowledge, has long been expressed by various

Kirkpatrick's four-level assessment model
Perhaps the most famous model for assessing the educational process isDonald Kirkpartick model The four-level assessment model, which was first presented in a series of articles in 1959 in theJournal of American Society of Training Directors (now known as T+D Magazine). This series was later compiled and published as an articleTechniques for Evaluating Training Programs in a book edited by Kirkpatrick, Evaluating Training Programs in 1975.

However, the four levels of Kirkpatrick's model only became popular after the release of his book Evaluating Training Programs in 1994. These four levels have become the cornerstone of the training industry in modern times.

While most experts refer to these four learning criteria as "levels," Kirkpatrick himself never used the term; he usually called them steps. (Craig, 1996). I will add that he did not call it a model, but used words such as “techniques for conducting the evaluation” to refer to it (Craig, 1996, p294).

These four steps consist of:

  • Step No. 1: Feedback or reaction (hereinafter in the text I will call the first level – Feedback) ( Reaction) – How much did students like/dislike the learning process?;
  • Step #2: Training ( Learning) -What did they study? (the degree of assimilation of knowledge and skills by students);
  • Step #3: Behavior ( Behavior) – What changes in the student’s work activity as a result of the learning process? (ability to apply knowledge and skills acquired in studies in work);
  • Step #4: Results ( Results) – What are the tangible (material) results of the training process in terms of reduced costs, improved quality of work, increased output, increased efficiency, etc..?

    Kirkpatrick concept is quite important because it is an excellent tool for planning, assessing, diagnosing problems, especially if we make minor improvements to it as shown below.

    Not just for training

    Some experts mistakenly assume that Kirkpatrick model can be used for training (here we mean formal learning processes: trainings, seminars – E.B.’s note), but the model could be used in working with other educational processes. For example, profession Human Resource Development (HRD) is concerned not only with the development of formal learning processes such as training, but also other forms such as informal learning, development, mentoring and nurturing (Nadler, 1984). The manual, published by one of the founders of Human Resource Development (HRD), Leonard Nadler (1984), uses Kirkpatrick's four-level model as the main model for assessing the educational process.

    Kirkpatrick himself wrote: “These goals (referring to his article) will relate to in-house classroom programs. Many procedures and principles apply to all types of learning activities, such as performance analysis, participation in external programs, programmed instruction, reading specially selected literature" (Craig, 1996, p294).

    Development of the four-level model

    Due to the “age” of the model, as well as with the advent of new, modern technologies, Kirkpatrick model is often criticized for being too outdated and simple. But nevertheless, even five decades after its presentation, there was no adequate model capable of replacing Kirkpatrick’s model. And I'm sure the reason why model replacements aren't happening is because Kirkpatrick was basically right, but he made a few small mistakes:

    Motivation, Not Feedback

    When a learner moves through the flow of a learning process, such as an e-learning course, an informal learning episode, or receiving assistance from a mentor in performing a work activity, he must decide where to focus his attention first. If the goal or tasks are assessed by him as important or feasible, then the student is normally motivated to achieve them (Markus, Ruvolo, 1990). But if the task seems to him weakly related to his activities, or exists only low probability successful completion of this task, accordingly, the motivation to complete it is low. In addition, research on Feedback evaluation generally shows that it is not a valid tool for assessing success (see the last part of the Critique article)

    It's different from wordsKirkpatrick(1996), who wrote that Feedback is the extent to which learners evaluate a particular learning process. However, the less important the learning material is to the learner, the more effort must be put into developing and presenting that material. Therefore, if the educational material is not relevant to the needs of the learner, then the educational material should “hook” the learner with fancy design, humor, games, etc... This does not mean that design, humor or games are unimportant; however, their use in the learning process is much more than just making the process fun, they help and promote the learning process. And if the learning material is built on the basis of clear objectives and design, then it helps the learner to bridge the gap between existing and required activities. Therefore, they (design, humor, games) must motivate learning - if this does not happen, then something went wrong in the process of planning and creating the learning process! If you suddenly catch yourself thinking that you are trying to hook students with all sorts of enticements (design, humor, games), you should probably reconsider the goals of your educational process.

    Performance, Not Behavior

    Performance is better than behavior because activity has two aspects: behavior is the means and the consequence of behavior is the results. And it is these results that interest us most of all as noted by Gilbert (1998).

    Turn it into a Best Model

    The model where the two most important items, results and behavior, are at the end is turned on its head because the importance of the items is imprinted in people's minds in the order in which they appear. Thus, we reverse the model and add the above and get:

  • Result ( Result) - What impact (consequence or outcome) will improve your business?
  • Activity ( Performance) – What should employees do to achieve the desired effect?
  • Education ( Learning) – What knowledge, skills, resources do they need to carry out their activities? (courses and classroom activities should be last on the list)
  • Motivation ( Motivation) – What do they need in order to learn and perform activities consciously?

    This model becomes a tool for both planning and assessment = diagnosing problems (Chyung, 2008):

    Goals (planning)

    Assessment level
    What are our company's goals for business development?

    results


    Did you feel the expected effect?
    What do our trainees need to be able to do to achieve these goals?

    Activity


    Have the acquired skills been transferred to work by the trainees?
    What new skills, knowledge and resources are required for trainees to be able to perform the activity?

    Education


    Have trainees learned the necessary skills and/or resources?
    What do learners need to learn and perform activities consciously?

    Motivation


    Are they motivated to learn and do the work?

    The corrected model can now be used for planning (left column) and estimation (right column). In addition, it can be used to diagnose problem areas in the learning process. For example, you know that employees have acquired the necessary skills in the training process, but do not apply them on the job, then the following problem areas become obvious (in this example, this is the Activity cell or the cell to the left of it):

  • There is something in the employee's work environment that limits his ability to use the skills he has learned; or
  • The very premise that these skills will lead to changes in performance is incorrect.

    The diagram below shows how the given processes in the revised model fit together

    Organization

    1. Results

    Summative assessment

    Activity

    2. Working environment

    Formative assessment

    Learning environment

    3. Training

    Formative assessment

    People

    4. Motivation

    Formative assessment

    As the above diagram shows, Outcome Assessment is most interesting to business management, while the other three levels of assessment (activity, learning and motivation) are the main ones for the company's training designer to plan and evaluate learning processes; and of course Performance Evaluation is also important for him, as it is the provider of business development goals.

    Level one – Results

    Results or impact

    Although usually more difficult and time-consuming than the other three levels, it provides information that is of great value: it proves the value of the learning and performance processes. However, using the Objectives/Planning/Evaluating model should make the process simpler and easier and you will have a clear picture of what you are trying to achieve. That is. When you start planning something, you are in a better position to understand how to evaluate it.

    Motivation, Learning and Performance are largely a matter of “soft” units of measurement (so-called non-financial metrics); however, decision makers who approve learning processes prefer outcomes (return on investment or impact). Jack Phillips (1996), who probably knows the four-tier better than anyone elseKirkpatrick model , writes that the value of information becomes significantly higher if we move from motivation to results.

    The above does not mean that the other three levels are useless; in fact, the benefit of using them lies in localizing problems in the learning process:

  • The motivation assessment informs you how well the learning process meets the expectations of the trainees (this measures how well the learning process analysis process has been developed). You can have all the other levels correct, but if learners don't see the learning and activity objectives, they (the learners) will probably not achieve the learning objectives.
  • A training assessment informs you how well the training process actually works to train employees (this assessment measures how well the design and development of the training process and material works).
  • Performance appraisal tells you how well the skills learned actually translate into employee performance (this measures how well the performance review process is running).
  • Outcome measurement informs you of what the organization gets in return for what it invests in supporting learning. Decision makers generally prefer results in hard units of measurement, although not necessarily in dollars and cents. For example, a survey of financial and IT managers showed that they consider the return on investment in training client-oriented technologies in both hard and soft units of measurement, although specific gravity more specifically in non-financial metrics (soft), such as customer satisfaction and loyalty (Hayes, 2003).

    Let me note the difference between “information” and “refund”. Measuring Motivation, Learning and Performance gives you information to improve and evaluate the learning process, which is largely the responsibility of the company's learning designers; while measuring Outcomes gives you a measure of the return on investment of the learning process that is more relevant to business leaders.

    Measuring Results can be done using a balanced scorecard approach (Kaplan, Norton, 2001), which shows the impact or return on investment from four perspectives:

  • Financial: Measurement in metrics such as ROI, which shows the monetary return on investment, or how effective the outputs of a process are. Financial indicators can have both “soft” and “hard” units of measurement of the result.
  • Client: improving the field where the organization builds its differences from competitors in attracting, retaining, building deep and long-term relationships with target customers
  • Internal: Achieving excellence by improving processes such as supply chain management, manufacturing, or technical support.
  • Innovation and learning: the learning process supports organizational climate change, innovation and individual development

    Level two - Activity

    This assessment includes testing the student’s ability to incorporate the skills acquired during the training process into work. This assessment can be done formally (testing) or informally (observation). It is determined by how they answer the question: Do employees use newly acquired skills in the process of work?

    It is important to measure activity because the underlying goal corporate training improve results by transferring new knowledge and skills to employees so that they then apply them in their work. Performance should be measured insofar as workers do their jobs; The measurement should usually be taken by someone who has a close relationship with the trainee: a supervisor, a trained observer, or an interviewer.

    Level three – Training

    It is a measure of how deeply learners absorb knowledge, develop skills, and change attitudes and attitudes as a result of participation in the learning process. Assessment of learning usually requires post-testing to determine what skills the trainees have learned during the process and what skills the trainees already possess.

    Measuring the outputs of the learning process is important from the point of view of validating the learning goal. Measuring learning typically focuses on the following questions:

  • What knowledge was acquired?
  • What skills were developed?
  • What settings have been changed?

    Trainee assessments are created to ensure the learner’s ability to perform an activity. There are two aspects to this process: the actual collection of information and data (testing the trainees) and the evaluation of this information (what does the data tell us?). This assessment should not be confused with learning assessment (evaluation). Assessment shows the progress and individual achievements of students, while Assessment of learning (evaluation) in general about the evaluation of training programs (Tovey, 1997, p88).

    Level four – Motivation

    Assessment at this level measures how learners perceive and respond to learning processes and activities. This level is measured using attitude questionnaires (motivational questionnaires), proposed after many training sessions. Trainees are most often well aware of what they need to complete the task. If the educational process did not satisfy their needs, then you need to decide: either this is a mistake of the educational designer (a specialist who designs the educational process in a company), or the students did not appreciate the benefits of the process.

    When a learner begins to study educational material, whether it is eLearning, mLearning, classroom training or social media learning, he must decide what he pays attention to in this material first.If the goal or tasks are assessed by him as important or feasible, then the student is normally motivated to achieve them (Markus, Ruvolo, 1990). But if the task seems to him to be weakly related to his activity, or there is only a small probability of successfully completing this task, then the motivation to perform it is low.

    Criticism

    There are three problematic assumptions of the Kirkpatrick model: 1) the levels are not arranged in ascending order (there is no hierarchy of levels); 2) the levels are not causally related to each other; 3) the levels are positively correlated with each other (Alliger and Janak, 1989).

    The only part of Kirkpatrick's four-level model that has not stood the test of time is Feedback (level one). For example, the Coach of the 21st Century school has one of the lowest indicators on the first level - the level of feedback, but is responsible for high performance activities (fourth level), which measure the productivity of their graduates. And this is not an isolated case: from study to study, the results show a very low correlation between Feedback and how well employees perform after receiving training (Boehle, 2006).

    Much more important than measuring feedback, as we have found, is preparing trainees for the learning process through conversations with the supervisor about the need to participate in training, followed by follow-up to ensure they have mastered the skills transferred in the training process (Wick, et al. 2006), and this is another reason why " Feedback“should be changed to “Motivation.”

    Kirkpatrick's four-level model is applied only after the end of the educational process. While the purpose of assessment should be to accompany the entire educational process, starting from the initial stage - the stage preceding the educational process.

    Actually, this criticism is not accurate. Eg,The ASTD Training & Development Handbook (1996), published by Robert Craig, includes chapterKirkpatrickwith the simple title “Evaluation”. In this chapter, Kirkpatrick discusses control groups, and before-and-after methods (such as pre- and posttesting). He goes on to discuss that the fourth level should include a post-training assessment three or more months after the end of the training process to monitor how trainees have begun to apply the skills learned during the training. Kirkpatrick further notes that Assessment should be included in all stages of the learning process, not only during each session or module, but also after each subject or topic.

    The four-level model only works for classroom learning, not other forms of learning.

    As noted in the “Not just for training” part (see above),Kirkpatrickwrote about the possibility of using a four-level model for other types of learning and HR, which helps ensure that both formal and informal learning processes are served, and whereKirkpatrick model serves as one of the main assessment models. And maybe the real reason that informal education advocates don't see the benefit of usingKirkpatrick models is that the four levels “were not invented for them.”

    The four levels of assessment have little relevance for other divisions of the company and lines of business

    One of best books for training and developmentThe Six Disciplines of Breakthrough Learning , written by Wick, Pollock, Jefferson, Flanagan (2006). They seem to put forward the most reasonable criticism that I have seen: “Unfortunately, this model is not widespread among business area managers who are directly involved with learning outcomes. Therefore, when heads of training departments write or speak in terms of assessment levelsKirkpatrick models with their colleagues in the business, rather than clarifying it, it often confuses the discussion and requires additional effort to build understanding between the business and the learning function.”

    And it is quite possible that this criticism is not essentially directed against theKirkpatrick models , but against those of us who build communications with business. We tell businesses: the first level shows how happy the trainees are from learning, at the second level they successfully passed the test, etc. throughout the model. Additionally, according to the research I've seen, learning outcomes are rarely used (the very thing that businesses value most). All other levels of assessment can only be useful in the process of designing learning experiences because they help us understand which type of assessment to use in which context. problematic situation. However, outside of this framework, these levels are of no interest to anyone. Most management is only interested in the impact of training and the answer to the question: do the resources we spent on employee training contribute to the development and prosperity of our business?

    • Term type: Definition

    Term: Kirkpatrick model

    Alternative terms: -

    Short Description: Kirkpatrick model - a four-level model for assessing the effectiveness of training

    Long Description:

    Kirkpatrick model- four-level model for assessing the effectiveness of training:

      Reaction (emotional level)

      Mastery (level of knowledge)

      Behavior (skill level)

      Result

    Reaction (emotional level)- emotional assessment of learning by participants . On the one hand, a positive reaction as such is not of great importance, since “liked the training” does not at all mean “ effective training" It may not change the participants in any way. On the other hand, this factor should not be underestimated: satisfaction with the training is the interest, attention and positive attitude of the participants, necessary for successful training.

    Mastery (level of knowledge) shows what knowledge, skills, techniques and methods have been acquired by the training participants. Acquiring new knowledge is an important, but insufficient result of training. To gain knowledge, it is not necessary to undergo training - just read a book or attend a lecture. In addition, the acquisition of new knowledge in itself, without the ability and desire to apply it, often does not affect the effectiveness of a person’s work. A more valuable result of the training may be that the participants receive “realizations,” so-called “insights,” when the training participant receives some knowledge not from the trainer, but as a result of analyzing the experience gained during an exercise.

    Behavior (skill level) - changing the behavior of the participant in a work situation, applying the acquired skills in work conditions. Behavior change shows that participants are putting into practice the knowledge and skills acquired during the training. This the most important indicator efficiency, since the main goal of the training is to improve business performance by improving the behavior of participants. After all, new knowledge and skills are useless if they are not applied.

    Result- Measurable results are identified (for example, improved quality of service, reduced waste, etc.). Companies usually have data on key, integral business indicators, such as sales volume, profit, costs. It is clear that the training provided is not the only factor that influences these indicators; there are many other factors, both external and internal. And it is quite difficult to accurately measure the contribution of training, for example, to increasing sales volume, especially if it was training on leadership skills or self-organization skills. The costs of researching this indicator can be many times higher than the cost of the trainings themselves, so this level of assessment is recommended only in the case of long-term and expensive programs.