Mapping
Research
Theory of Change
Impact > Simplified
Evaluation
Evidence
Insights
Reflections, opinions, explainers, related to evaluation.
You don’t have to know anything about evaluation to understand the content.
Have a cuppa, read an article, let me know what you think.
Overcomplicating an Intervention
Often social issues are complicated. Fully addressing issues often means we want to address lots of things at the same time. But this leads to a problem. It ends up trying to do lots of things, but not well.
I saw an example of this, just this weekend.
Submissions to the HEEL: how to be brave and balanced [Crosspost > TASO]
Last week, I wrote a guest article for The Centre for Transforming Access and Student Outcomes in Higher Education (TASO) about their upcoming Higher Education Evaluation Library (HEEL).
Being Carried By a Wave
In a busy Half Marathon, you set off in ‘waves’. As I was waiting for my race to start, I wondered: “Would being in a faster ‘wave’ also make me faster?”
This is “Regression Discontinuity” Explained Though a Half Marathon
Improving the Office for Students’ Standards of Evidence
The OfS Standards of Evidence are simple. They suggest that each of the 3 ‘types’ aren't meant to be ranked. But in practice, that isn’t how it works.
I think this is a problem, and one with an easy fix. This is my simple way to improve it.
Do you need to learn RStudio?
I work in Research and Evaluation, with lots of data. When I meet those who specialise in the big quant stuff, they tend to use RStudio.
So, for a while, I’ve been battling with one key question: Do I need to learn to use RStudio?
My choice took me a decade. Clearing students have days.
It took me a decade to shave my head. Right now, thousands of Clearing students are making a life-changing decision in days. And universities have almost no idea which of those students are ready.
The assumptions behind the ‘Hard Hat and the Graduation Cap’
I am broadly in favour of the idea, but I think there are some massive assumptions being made. And as we know, assumptions floating about in your Theory of Change are a big problem.
Does Manchesterism work in Mansfield?
Evaluating Graduation
We’re in graduation season at Nottingham Trent University. I’ve been thinking “how do we know this worked?”. It seems like a simple question doesn’t it.
5 Films that illustrate counterfactuals, but differently
Humans are obsessed with ‘what if’.
These 5 films, demonstrate 5 different ways to explore the counterfactuals, and 5 different evaluation methods that could be used to replicate them, when you don’t have a Delorean.
Did my swanky new shoes CAUSE my PB?
Last Saturday, I put on my new running shoes and ran my Parkrun PB #Humblebrag
The question is, was this down to the new shoes? Was it my ‘intervention’ that caused the success?
It’s your classic evaluation question.
We get ‘Sense of Belonging’ wrong
We’re obsessed with trying to get students to feel a sense of belonging at uni. It’s often something that we are trying to make happen, and we measure whether it has with a survey.
What if belonging isn’t an outcome at all? What if belonging is a ‘change mechanism’.
Kids in the Shire get left behind
I grew up in Shropshire, where there are rolling hills and places to buy homemade jam. It also has some of the lowest rates of access and participation to higher education in the country.
And it’s going to get worse.
Data doesn’t tell stories. People do.
The 2026 local elections happened last Thursday, and throughout the weekend, every pundit under the sun has been crafting their stories… but a lot of these are completely conflicting.
They all have access to the same objective data in the form of votes. But they are coming to different conclusions. So what’s happening?
What happens to evaluation when universities are skint?
The HE sector is in trouble.
But, when universities are fighting for survival, can you really trust what their evaluations say?
Outputs vs outcomes… and driving through France
Organisations and businesses often make a key evaluation mistake: conflating outputs and outcomes.
And for organisations and businesses across multiple sectors, it is incredibly costly.
6 mistakes people make when designing survey scales
Creating your own survey question seems easy enough, but I’ve seen some absolute whoppers in my time.
Let me show you a bad example, with 6 mistakes I’ve seen in real life, and what to do to fix it.
They spent two years finding out something ‘didn't work’. That’s great news.
A two-year study. Three universities. Randomised controlled trials. And the result?
The intervention didn't work. But that isn’t the end of the story.
The Romans drove on the left. And we didn’t have to make a survey to find out.
If you want to find out what side of the road that a Roman drives on, you can’t survey them. But in 1998, an old Roman quarry was uncovered near Swindon and it gave the answer.
How do you find out about behaviour without speaking to anyone?
Is this car ‘good’…?
Evaluation can be about assessing the value of something. But value isn’t just one thing. Understanding this has implications for what we evaluate, and the ethics behind it.
So, let me explain it simply, by asking “is this car good?”.
Is evaluating Parkrun a huge waste of time and money?
It’s obviously common sense that Parkrun is good for your health. Spending money evaluating it is a waste of taxpayers money, right? ….what a load of nonsense.