Assessment software; Best to do it yourself?

Blog post by Chris Peat, Customer Services Director, Axia Interactive Media and Board Member of the eAA.

We all know how important it is to assess someone’s skills accurately. The result of an assessment has the potential to be highly motivational for the individual and conversely it can have a crushing effect. Why then are organisation content to adopt ‘do-it yourself’ online assessment systems and sites. Across the country there are internal technical teams making the case that they have skills to create an effective online assessment system, using one of the ‘kits’ that are openly available.

I wonder whether any of us would buy a car which we know had been built using a ‘Do it Yourself’ kit rather than in a factory. Of course it may well work as it should and even look the part but would we really take the risk?

There are certain reassurances that come from buying a factory built car. One of the key ones is that it matters hugely to the manufacture that it works because their reputation depends on it. Witness the impact on a manufacturer when cars are recalled when an issue is identified, even though the impact is mitigated in part by the fact they can fix the fault. It also matters that you know that the parts to fix any fault are readily available and there is an authorised and fixed development methodology.

The reality is many of the ‘do it yourself’ kits are not made by anyone, indeed apparently the fact that they are a product of co-operative endeavours is to their advantage. However, imagine a situation where the reputation of an institution was badly damaged, when it was discovered that the assessments they had made, using their own internally developed solution, were fundamentally flawed. Aside from sacking their Head of IT who else could they pursue and who would have to take responsibility for fixing the issue?

It is true that there are lots of pressures to avoid a reliance on a supplier of proprietary software not least because companies have been guilty of sometimes overselling their software. Dishonestly answering the question ‘Can your software do this? with the standard ‘Yes’ when they know it is not fundamentally designed to do that.

In this respect the top prize must go the person from ‘Salesforce’ who apparently managed to persuade the now defunct College of Social Work that their software could support online professional development. 

In the end though suppliers have some culpability here for overselling their wares; ultimately it is the responsibility of those who assess skills to select software that delivers a high quality and reliable assessment and does not short change the person being assessed.