Benchmarking is a self-improvement tool allowing organisations to compare themselves with others regarding some aspects of performance, with a view to finding ways to improve current performance. Benchmarking deals with change for quality enhancement but also with identification and implementation of areas of development. Benchmarking is a process that enables comparison of inputs, processes or outputs between institutions or within a single institution over time
A Dual Modes Distance Learning Benchmarking Club was set up across the world, aimed at disseminating and implementing the Pick&Mix model. Four institutions in the Club were successfully benchmarked using a slightly modified version of Pick&Mix during 2009-2010. Besides the benchmarking, concordance works with other benchmarking methods were conducted.
The Pick&Mix benchmarking system contains 18 criteria which are critical success factors (of special importance for success in e-learning).1 These are: Usability, e-Learning Strategy, Decisions on Projects, Training, Costs, Planning Annually, Technical Support to Staff, Decisions on Programmes, Leadership in e-Learning, Management Style, Relationship, Management Upwards, Reliability, Market Research, Security, Student Understanding of System, Student Help Desk, Distance Learning Strategy (doppelganger criterion with e-learning strategy). All of the benchmarks were scored at six levels (a scale of 1 to 6) and a coloured matrix was produced where the state of an institution/department becomes explicit.
Among four institutions in the Club that went through to scoring, one was in UK (University of Leicester), two in Sweden (Lund University and the Royal Institute of Technology), and one in Canada (Thompson Rivers University). Later, the University of Leicester decided not to continue in the Club, due to changes in staff and policies. Fortunately Gotland University, Sweden had also done benchmarking using the Pick&Mix methodology. A final meeting of the Club was hosted by KTH in September 2011 with Thompson Rivers University and Lund University with Gotland University as a guest. The four universities present agreed to share their results and to continue their benchmarking collaboration in Sweden and beyond, including producing a public report of their conclusions. This paper is the first airing of these.
Headline conclusions are:
1. Over the five years since Pick&Mix was first used, institutions are gradually developing stronger competence in e-learning – but progress is slow
2. Only a very few universities have strong expertise in market research for e-learning
3. No institution has a good understanding of the cost issues to do with e-learning
Staff generally feel that the management do not given them adequate reward and recognition for their e-learning competence.
1 Re.ViCa http://revica.europace.org
2012. 91-103 p.