5 Films that illustrate counterfactuals, but differently

Humans are obsessed with ‘what if’.

There are moments in life which cause your life to go down one path, but what if it went down the other? There’s ‘what happened’, and then there is ‘what would have happened’.

The latter is the ‘counterfactual’. The ‘what if’.

Evaluation is all about that ‘what if’. About finding and understanding the counterfactuals. But evaluators also understand that not all ‘what ifs’ are the same question. They appear in different ways.

And because humans are obsessed with the ‘what if’, there are countless films, tv shows, books, songs, etc. that explore all of these different counterfactuals. Practically all of Sci-fi is based on it [and maybe why I’m so into sci-fi!]

Lots of these give different lessons, but I’m going to narrow it down to 5 films.

These 5 films, demonstrate 5 different ways to explore the counterfactuals, and 5 different evaluation methods that could be used to replicate them, when you don’t have a DeLorean.

1.     Sliding Doors: the classic ‘what if’

Helen misses her train, AND she catches it. The film splits into two and you see both lives running in parallel.

Same woman, but two timelines and two very different outcomes.

This is the classic version of a counterfactual in modern cinema. We even talk about ‘Sliding Doors moments’. It’s also structurally, close to the logic behind a randomised controlled trial (RCT). One single isolated event, a control group and a treatment group, and all the effects stem from that.

Everything is the same, except whether she catches the train.

Of course, in real life we can’t map the effect of a single change like this.

Evaluators spend their careers building stand-ins for this effect. This is a clean binary. Intervention vs Non-intervention. Caught train vs missed train.

What we often turn to here is a randomised controlled trial (RCT) to replicate the effect. If a young person ‘catches’ our intervention, how do their outcomes change…?

2.     It’s a Wonderful Life: same trick, different idea

George Bailey stands on a bridge, and wishes he’d never been born. So Clarence, his guardian angel, grants him that wish. George gets to walk through Bedford Falls, which has now come ‘Pottersville’, and sees exactly what happens to the town and everyone in it, if he didn’t exist.

At first glance, it looks just like sliding doors. See what would have happened in a different situation. And methodologically, it is basically the same. Still a counterfactual, and still a clean binary.

But there is a key difference in the change. Unlike Sliding Doors, which changes an event, this is removing a person from the equation. In widening participation, we often think about this tension. How much of a person’s outcome do we attribute to the programme they took part in, versus the individual mentor or teacher or role model?

Formal interventions are countable. But relationships are harder to count.

The whole point of this film argues that the presence of one person mattered more than any intervention or money or thing that happened in Bedford Falls.

Here we can’t just think about an RCT, we need to start considering relationships, the impact of a person. At this level, we have to think about not just WHAT happened, but HOW it happened.

So what about the why…?

3.     Back to the Future: Moving from what to why

Marty McFly goes back in time, and doesn’t just see his parents relationship happen, he breaks it. Then he has to figure out exactly how it happened, and how to recreate it.

Again, on the face of it, we can trace the whole relationship back to a causal incident: George McFly getting hit by a car because he is being a ‘peeping Tom’. That’s the first bit of the film.

But the rest of it is about trying to understand how and why that worked, in order to recreate it. Marty needs to understand the mechanisms underneath the event.

It wasn’t really about the car. It was about an interaction between George and Lorraine, building a bond between them. And once Marty understands the mechanism, he doesn’t even try to recreate the original intervention. He creates a different and better moment, that has better outcomes.

This isn’t process tracing or RCTs… we’re getting into the territory of ‘realist evaluation’. We want to understand the mechanisms and the context, how they combine so you can reproduce, or improve on it.

For a young person, we can repeat our interventions again and again, but it’s more powerful to understand the reasoning behind what sparked their outcomes, rather than rinse and repeat. I wrote about this in a previous blog about belonging. If we see belonging as a mechanism rather than an outcome, then we see this as a way to make changes happen, and not try to recreate conditions again and again.

And once we start thinking in terms of mechanisms rather than events, a new question is raised: what if we don’t want to reproduce one good outcome, but want to keep refining it and expanding on it…?

4.     About Time: Getting better on purpose

Tim finds out that he can go back to any point in his own life, and start to make changes. This isn’t about one moment, but small adjustments to many and seeing what happens.

It’s not treatment vs counterfactuals, it’s lots and lots of treatments and counterfactuals, learning and improving through repetition and small changes.

This is developmental evaluation: No fixed endpoint, no single final verdict on success or failure. Just lots of continuous adjustments based on what the feedback loop tells us.

It’s not “did this work”, but “what did we learn, and what do we do differently next time”.

This is also close to the spirit behind the updated Magenta Book, with its emphasis on ‘Test and Learn’ as a formal approach, rather than something you fall back on if your RCT doesn’t fully work. It’s deliberately incremental.

For a young person, this isn’t about trying to create a fork in the road for them. It’s about trying to slowly build and improve a way of learning, or studying, or building interests or skills or identity, through lots of methods.

These iterations may seem small scale. But zoom out far enough, and we stop asking “what should we try next”, and start to think bigger.

We can ask “what does the system look like, how does it interact, and how can we leverage this to get specific outcomes…?”

5.     Everything Everywhere All at Once: The whole system, at once

Evelyn doesn’t have a time machine or a rewind button… she gets everything! Every possible version of herself, all at once, all interacting, all possible and accessible to one knackered woman in a laundromat.

In order to do this, we don’t just iterate and test and learn. We have to understand the system. A person sits inside layers of family, relationships, history, circumstances, choices, a network…

This is close to ecological systems mapping. A young person’s outcomes aren’t produced by one intervention, or one relationship, but by interacting layers of family, school, peers, community, policy, place. If you want to create real change for a person, it requires systems thinking, and systems change.

This is messy, this is hard, because its about understanding everything everywhere all at once, in order to fully understand and affect the change you want.

Having said that, no amount of systems change will give you sausage fingers or turn you into a rock, so take this particular film with a pinch of salt!

Back to reality

I love films that deal with counterfactuals. There are so many more of these I could have gone into. And I didn’t even touch on TV shows like Man in the High Castle or For All Mankind (which are GREAT).

But all of these things get to cleanly illustrate a counterfactual. We see the ‘what ifs’. And sadly in life, we don’t get that. There is no Clarence for widening participation, or a Delorean in your Access and Participation Plan.

We have to work with proxies, comparison groups, and controls. It’s much less satisfying than the films, but it’s the closest we have. And when it’s done well, we get stories that are incredibly powerful. And they’re real.

Anyway, I hope that’s got you thinking about counterfactuals. I’d love to hear your favourites!