How should we measure success?
One of the first questions I often ask when discussing a potential learning project is: “How will we know if this training is successful?”
For me, that question belongs near the beginning of a project, not at the end. If we can agree what success should look like before we start designing, we have a much better chance of creating learning that contributes to it.
That sounds obvious, but training is still often evaluated using the measures that are easiest to collect. We know how many people attended or completed a programme. We know whether learners enjoyed it. We may have assessment scores showing what they knew at the end.
All of those measures can be useful. The problem comes when they become substitutes for the more important question: did anything change that mattered?
Start with why the training is needed
The question “How will we know if this worked?” is closely connected to another: “How do we know we need training?”
Recently, I spoke with someone whose organisation had identified a lack of awareness around an important part of the business. They knew the problem existed because interviews and mystery shopper research had given them evidence of it.
My immediate thought was that those same measures could potentially help us evaluate the programme afterwards. If awareness improved, could we see that change in another round of interviews or mystery shopping?
In this case, repeating the research was too expensive. That was entirely understandable. Evaluation should be proportionate to the investment being made.
But it still left us with an important design question: if we cannot repeat the original measure, what other evidence would give us reasonable confidence that the programme had made a difference?
That conversation is valuable in itself. It encourages everyone involved to be clearer about the change we are actually trying to create.
The right measure depends on what success means
It is easy to dismiss measures such as completion rates because they do not prove that training has changed performance. But I don’t think the answer is to create a hierarchy in which some measures are always “good” and others are always “bad”. Their value depends on what we are trying to understand.
Consider health and safety training. After a programme, an organisation might see more reported near misses, not fewer. At first glance, that could appear to be a poor result. But if employees have become better at recognising near misses and understand why reporting them matters, an increase can indicate exactly the behaviour the programme was intended to encourage. The incidents may already have been occurring; the organisation simply has better visibility of them.
Completion or usage data can also tell us something meaningful in the right circumstances. We have worked with learning activities where AI gives people an opportunity to practise a particular type of workplace conversation and receive feedback. If learners return to that practice repeatedly, the number of visits becomes interesting. It does not prove that their real-world performance has improved, but it tells us that people are choosing to practise.
If that activity remains available when people need it, we might also think of it as performance support rather than simply a course to be completed. In that situation, somebody returning to practise shortly before a real conversation may be exactly the behaviour we hoped the resource would encourage.
What should we actually measure?
The familiar Kirkpatrick Model offers four useful perspectives: Reaction, Learning, Behaviour and Results. Kirkpatrick’s current approach also emphasises beginning with the organisational results we want and working backwards.
Each level answers a different question. Learner reaction can tell us about the experience. Assessment can tell us whether learning occurred. Behaviour measures can help us understand whether people are doing something differently at work. Results look towards the organisational outcomes the intervention was intended to support.
What matters is not collecting every possible measure. It is choosing evidence that makes sense for the purpose of the programme.
That is also why I am cautious about treating a high learner satisfaction score as proof of effectiveness. Some worthwhile learning experiences are challenging. They may surface different perspectives, encourage people to question established practice or generate conversations that continue afterwards. A programme can be useful without being universally comfortable.
In practice, the most convincing picture often comes from a small collection of relevant evidence rather than one perfect metric.
Evaluation can make the learning better
There is another reason to have this conversation early: evaluation can influence the design itself.
High Impact Evaluation takes this further by looking not only at how learning is evaluated, but also at the factors that may support or prevent transfer into workplace performance. Its approach includes aligning learning with organisational outcomes, identifying risks before launch and looking for indicators of likely impact.
That encourages some useful questions during design. If we want someone to behave differently after a programme, what might get in the way? Will their manager support the new behaviour? Do their systems and processes make it possible? Are they likely to have opportunities to practise? Are other organisational pressures pulling them in a different direction?
Those are not simply evaluation questions. They are questions about whether the training has a realistic chance of succeeding.
And that, for me, is the real point.
A strong evaluation plan is not an extra layer added to a finished training programme. It is part of good learning design. When we are clear about the problem, the change we want to see and the evidence that would give us confidence that change is happening, we can make better decisions about what to design in the first place.
It also gives L&D and HR teams a much stronger story to tell afterwards. Rather than reporting only that 95% of people completed the programme or that most learners rated it highly, they can explain what success was intended to look like, what they chose to measure, and what the evidence is telling them.
That is a much more useful conversation about the value of learning.
1. What change are we trying to create?
2. What evidence would show us that change is happening?
3. What is realistic and proportionate to measure?
Want to explore working with us?
If you'd like to work with us, click the button below to get in touch.
