Time for a paradigm shift in training evaluation? - The systems thinking and training blog - Bryan Hopkins

Go to content

Main menu:

Time for a paradigm shift in training evaluation?

Published by in Evaluation ·
The history of modern training started in the 1940s, when the demands of a wartime economy made it essential to train large numbers of people in the same skills in a short period of time. Up until that time training relied essentially on ‘sitting by Nelly’ activities, where a trainee sat with an experienced person who could monitor their progress and decide when they were ready to play a full part in the workplace. By the 1950s modern training was well established, but it became apparent that there was a need to find new ways of seeing how effective it had been. Enter Donald Kirkpatrick.

Kirkpatrick established a framework for evaluating training which is familiar to everyone with the slightest connection to training. Half a century after it was established, it is still the standard approach for thinking about training evaluation activities. And yet, practitioners constantly discuss the difficulties with using it. Industry data shows that few evaluation projects work through the full four levels. A succession of academics and practitioners have tinkered with it, adding levels and refinements here and there in attempts to make it more relevant to contemporary practice.

One of the problems is that it is really a framework, a taxonomy rather than a model. It has proved very useful for identifying different levels at which evaluation can take place, but is less effective at providing a method for carrying out the evaluation. For this, evaluators have to draw on their own bags of data collection and analysis tools. Also, of itself it does not encourage the evaluator to think carefully about the learners and their world apart from how they are reacting to the training.

As such, it is perhaps a product of its time. In the 1950s the United States had a booming industrial sector, employing large numbers of people in manual trades working on relatively simple technologies which were not evolving that quickly. Apart from the shadow of the hydrogen bomb, it was a bright and optimistic time and people were, in general, not questioning the social, environmental and political context within which they lived.

However, now in 2019 the world is very different. The United States and Western Europe have moved into post-industrial economies, many fewer people are involved in manufacturing industries and the new emphasis is on the ‘knowledge worker’. Technology is changing very rapidly, and employment patterns are adapting commensurately: people move in and out of organisations much more quickly and nobody expects a ‘job for life’ with a single corporation. Politically, these countries are also much more volatile. The gentle oscillation between more or less right-wing and more or less left-wing political parties has been replaced by a lurch to the right and the questioning of liberal democracy, which in turn has politicised these societies to a degree not seen for many years. Not unconnected with this is the climate emergency, the growing realisation that current economic patterns are not sustainable and that radical change is needed.

Current economic systems are, like the Kirkpatrick framework, a product of the baby boomer generation, but this politicisation is increasingly centred around the Millennials and Generation Z. Particularly in the case of the climate emergency, these are the people with the most to lose. From this perspective, a major problem with a Kirkpatrick-based way of evaluating training is that it assumes that employees are all lined up and as one with their employer’s view of the world, keen to play their part in increasing profitability and shareholder value. But is this still true?

Earlier this year, employees at Google decided that they were not prepared to work on Project Maven, a project integrating facial recognition systems into military drones. Over 3000 employees signed a letter to Sundar Pichai, the CEO, and apparently over a dozen employees have since resigned. In August the Business Roundtable, a US-based organisation representing employers, released an open letter signed by over 180 chief executives stating their commitment to protecting the environment by embracing sustainable practices. Then, on September 20th, millions of young people around the world walked out of schools and workplaces on strike in protest at the climate emergency. Young people are no longer prepared to switch off their moral standards in order to find or keep a secure job.

So how to think about this? The German philosopher Juergen Habermas has a particular interest in how power works in society. The diagram below is a representation of Habermas’ ideas about power relationships.


Habermas’ conceptualisation of power relationships (taken from Field, 2019)

The corporate world is above the dotted line, where money and power determine the technical interest of organisations. Below the line is the life world of employees, shaped by their practical interest (ethical priorities, work-life balance needs and relationship hopes) and an emancipatory interest (about maintaining control over one’s life). There is a tension between each individual’s practical and emancipatory interest, but also crucially, between these and the organisation’s technical interest. Once upon a time, it could be assumed that this technical interest would prevail, but perhaps this is no longer a safe assumption?

So what are the implications of this for evaluating training? What it may mean is that a factor to consider when looking at how training may contribute to organisational effectiveness is whether it satisfies the personal and ethical interests of employees. Increasingly, employers may find that if they ignore these issues, that the best people move on to places where they may find fulfilment. The Kirkpatrick framework may not be dead just yet, but it is certainly time to remember these prescient words:

Come gather around people, wherever you roam, And admit that the waters around you have grown
And accept it that soon you'll be drenched to the bone
If your time to you is worth savin'
Then you better start swimmin' or you'll sink like a stone
For the times they are a-changin'
Bob Dylan, 1963

Field, L. (2019) ‘Habermas, interests and organizational learning: a critical perspective’, The Learning Organization, vol. 26, no. 3, pp. 252–263.

No comments

Copyright 2015. All rights reserved.
Back to content | Back to main menu