What happens to evaluation when universities are skint?
I don’t know about you, but it feels like we’re in limbo.
Officially, we’re not in a recession, and there are signs that the Government have steadied the ship on the economy. But still, things feel… not good.
The OfS have reported ‘significant challenges’ for Higher Education, with nearly half of all UK institutions facing a deficit this academic year. John Blake’s paper for the Post18 Project posted last week describes how this financial crisis isn’t just down to the current economic wobbliness of the country, but is far more systemic, dating back 30 years.
In short, the HE sector is in trouble.
And now, universities need to demonstrate value to regulators, students, the general public… basically everyone… more urgently than ever before.
And evaluation can do that.
But, this situation raises a key question: when universities are fighting for survival, can you really trust what their evaluations say?
‘Failure seems to be the hardest word to say’
I recently spent almost an hour talking with Professor David Stevenson, one of the researchers behind FailSpace. I’m going to link everything at the bottom of this page, because it’s so worth a look.
Their research involved more than 200 people across the arts sector and explores how that sector talks about evaluation and increasing participation. Crucially, how are success and failures shared. It was published as a peer-reviewed paper in the International Journal of Cultural Policy in 2021.
The headlines are wild. Data consistently suggests a failure to address inequalities in participation, yet the bulk of evaluation reporting celebrates successes of participatory projects.
The data shows it isn’t working. The evaluations say it is.
The parallels between the Arts and Humanities sector, and the Widening Participation sector, are uncanny.
This isn’t about dishonesty
The story here though, isn’t that people are cynical and dishonest. For what it’s worth, I believe that, generally speaking, people are inherently good and trying to do the right thing. Particularly those that work to address social inequalities.
This problem is structural.
The whole landscape doesn’t encourage organisations to be truly honest. Organisations who describe success are rewarded, and those that describe failure, risk losing funding or damage to their professional reputations.
Besides, we all want this to work.
We want to show that addressing inequalities is a good thing, the work has impact, and is worth funding. Our programme hasn’t worked this time but keep funding us and we’ll get it right! Surely, we should pivot towards looking at the positives, because it’s better in the long run…?
If something is written in a neon sign, it must be true…
The thing is, by protecting the narrative of success, we also protect the conditions that produce the original inequalities in the first place.
Why is this worse now?
Funding for Access and Participation according to the OfS was £273.3 million in the 2025-26 budget, down 9.2% from the £301 million allocated in 2024-25.
So the stakes are massive, but the resources are shrinking.
Economic instability tips evaluation away from ‘learning’, and towards ‘proving impact’. This is something that has been building for years. Like John Blake wrote about in his paper, these issues aren’t just down to money. This evaluation crisis is also decades in the making.
The pressure has been ramping up with the introduction of OfS standards of evidence, with the introduction of What Works centres, with the increasing pressure on achievement in Access and Participation Plans.
I’ve worked at a ‘What Works’ Centre, and it was filled with brilliant people, who genuinely want to do evaluation right. So why is this adding to pressure?
The core problem is, and has been for a long time, that there is a reward for showing impact, but there is no reward for showing failure. And there needs to be. Because as economic instability grows, the pressure for universities to show success grows with it.
There would also be massive value in a ‘What Doesn’t Work’ Centre, but those are a harder sell.
What we need more than ever is to learn from failures.
So what do we do?
We evaluators have a role. FailSpace have produce toolkits and a body of practice designed to help sectors build a culture of learning from failure. The things we also need to remember:
Design your evaluation around successes and failures. If your evaluation framework only has space to measure and discuss success, then that is what it will do. But we need to also ask ‘what didn’t work?’, or ‘who hasn’t this reach’, and ‘what do we do differently’, from the start.
Separate learning evaluation from reporting impact. They serve different audiences and different needs. Evaluation and monitoring and communicating impact stories are not the same thing. When an evaluation gets used for everything, it all gets corrupted
Record everything and share it with others outside your organisation. Completely suppressing failure doesn’t just mislead a regulator, it harms the sector. It perpetuates those conditions for inequality, and one organisation cannot fix these issues alone. Building an honest network is critical.
But there also needs to be a top-down change too.
Regulators need to reward organisations who evaluate well, show a failure, and a plan for what they will do differently. TASO is about to bring out the Higher Education Evaluation Library (HEEL) and have a massive opportunity to change the game. Promote and reward those institutions who have null results and a plan to address it.
So where does that leave us?
Economic instability doesn’t kill evaluation, or stop it happening. It corrupts it. With the best of intentions.
Towards the end of my conversation with Professor Stevenson, we started getting philosophical. Embracing failure is something you don’t just do in evaluation, but in life.
Everything is a mixture of successes and failures, and we need to reflect that in our work.
If someone reports only the success of something, or describes something as a failure without learning, then I question whether you can trust it.
An honest evaluation is not a luxury for when the HE sector is swimming with cash. It is the only way for things to get better. So don’t just try to report success, embrace failure too.
Further Reading:
The FailSpace website is here and the paper is here. [Both are worth your time!]
Jancovich, L. and Stevenson, D., 2021. Failure seems to be the hardest word to say. International Journal of Cultural Policy, 27(7), pp.967-981.
John Blake’s paper ‘Blood, Debt, Toll, and Arrears’ is found here. [Also worth your time!]
Blake, J., 2026. 'Blood, debt, toil, and arrears', The Post-18 Project. Available at: post18.co.uk
