AI allows us to assess differently – and we should

A couple of months ago you would have been forgiven if you’d never heard of Language Learning Model Technology or Natural Language Processing. Today, you are unlikely to have missed the controversy surrounding Open AI’s ChatGPT, which has exploded from relative obscurity into mainstream ubiquity and caused the world’s edu-twitterati to panic about humanities exams, the death of essays and the potential for widespread cheating.

ChatGPT certainly could be used by disingenuous students to cheat on homework (ultimately to their own detriment) and thereby create undiagnosed learning gaps. But the reality is that this technology won’t be catastrophic to current models of education delivery. On the other hand, simply trying to ban it, resist it or live in ignorance of it will be counter-productive. The train has left the station, and the technology is already far better than it was even two months ago thanks to humans (including its critics) experimenting with it and training it.

There is a debate to be had around the ethics of how students use it, but the reality is that at present there are severe limitations to what it can do, not least that it makes up quotes and references and that it makes mistakes. If it can achieve a decent pass in spite of these issues, this raises serious questions about our assessment systems.

More fundamentally, as these tools improve it will become increasingly clear that recall and regurgitation are not and will not be the most valued human skills. The pandemic has already shone a light on the inequity and weaknesses of assessment systems that value these skills above others, and reforming them must start by asking how artificial intelligence can support human intelligence.

One of assessment’s key purposes is to provide a currency to students when they leave formal education, and the risk to systems which focus solely or heavily on high-stakes assessments is that they could develop graduates with lower-value currency than those who can demonstrate a broad set of valued skills.

As such, the UK should be worried. It is among a group of countries that aced the PISA OECD rankings for systems where memorisation is the top strategy for passing exams and where elaboration is utilised the least.

But a broader suite of assessments under the same conditions – perhaps with open books or limited access to the internet – is not an adequate response. We must find a place for oracy, communication, presentation and discussion skills among others.

The closest we come to evidencing skills is asking students to write self-appraisals like the ubiquitous personal statement. However, these are often coached by an experienced teacher or careers adviser for those who can afford it.

I asked ChatGPT to write one for me. The result was dull and wouldn’t pass muster with particularly competitive universities – yet. But if software like ChatGPT is able to flood the median level of university and job application standards with generic content, it will make the sifting process all the more challenging, particularly for students and young people who have less support or resources to draw on.

UCAS has just published a report saying that the personal statement will be reworked to make it fairer. The jury is out on whether changing the format will level the playing field, but others have identified the same problems and are already working to get ahead of them.

‘Skills capture platforms’ like digital learner profiles and our own Story aim to capture and validate broader experiences and skills. Meanwhile, schools from Aiglon in the Swiss Alps to Avanti Schools Trust’s schools across England are using character education as a tool to combat the unnecessary pressure to game the system caused by traditional assessment methods.

These efforts to demonstrate students’ holistic record will allow universities and employers to see evidence of skills progression and complement the personal statement. The challenge for policy makers, educators, technologists and entrepreneurs is to get beyond these piecemeal solutions to a national solution that leverages technology to increase equity for all. Change is happening, and the guarantee of inertia is worse than the potential of failure.

We end our series of expert opinions on AI’s impact on education with two contrasting opinions about assessment. Read the other here.