Outputs vs outcomes… and driving through France
Organisations and businesses often make a key evaluation mistake. It goes like this:
An organisation runs a programme or delivers a project. They track ‘deliverables’… attendance numbers, sessions delivered, hours logged… all the targets are met. Then 12 months later, a senior leader or funder asks what the impact was.
The honest answer is that we don’t know. KPIs and targets were met, but the impact is still unknown.
This isn’t a project management problem.
This is the problem of conflating or not distinguishing between outputs and outcomes. And it for organisations and businesses across multiple sectors, it is incredibly costly.
Lessons from driving through France
13 years ago, I drove Molly, my 1997 Nissan Micra, from Sheffield to the Vendée in France, for a holiday with friends.
If you haven’t met Molly yet, I introduced her in a previous blog post. She was my first car, a tiny, super reliable, completely batter purple Micra. 1.0 litre engine, no air con, no CD player, no power steering, and with seats designed by someone who may have read about human spines but apparently never owned one.
This was a big one for me.
It would be my longest road trip, the first time driving on the ‘wrong side of the road’ (which the Roman’s agree with me on), and the first time navigating different road rules and road signs.
Seriously though, the seats were horrendous….
If you were evaluating that trip on outputs alone, you’d probably measure… Miles driven, Litres of petrol, Hours taken, Haribo eaten, Amount spent at French tolls…
All easily measured, but all misses the point. The outcomes of the trip hint at the wider impact it had on me.
Belonging and social: I had a lovely holiday with my friends and made memories that lasted.
Confidence and self-efficacy: I came back with confidence behind the wheel that I didn’t have before.
Behavioural and skill development: I’ve driven through France loads of times now, and I love it! There is a skill increase, as I can navigate the French road system (which is actually much more pleasant than the UK).
Physical: I also had spectacular back ache because the seats were terrible and the steering was heavy. Not every outcome is positive, and good evaluation looks at this too.
So what is the difference?
An output is something that has been produced.
Something that tangibly happened. Session delivered. People reached. Documents created. Miles driven.
An outcome is what changes as a result.
Behaviours changed. Knowledge gained. Confidence built.
Outputs are easier to measure, as you can often just count them, and put them in a spreadsheet. But they only tell you what happened, not what mattered.
You can hit every output target and still have achieved nothing of lasting value.
The tech industry is experiencing this full on, right now.
I was speaking recently to my (soon to be) brother-in-law, who knows the tech industry inside out. He has strategic responsibility for delivering tech solutions to businesses, and trying to solve this exact problem.
A business introduces a new system. The project runs on time, every deliverable is met, the software is launched. Every output is ticked.
Then… not much actually changes.
Adoption is low, people find workarounds or ignore the new system. The business has spent serious money, and nothing has fundamentally changed.
This isn’t a tech problem.
It’s about understanding outcomes, and adoption. Projects are designed around outputs, but nobody fully asked:
“what would it take for people to change behaviour?” or
“what is this new tech going to change for the user?” or
“what behavioural outcomes are we expecting from this project, and how are we going to get there?”
Without this, you can build something perfect that nobody uses. And with AI suddenly being quote-unquote ‘needed’ everywhere, this problem is happening all over.
Back to France, and what if…
So, Molly, France, 13 years ago.
Let’s suppose someone looked at my trip purely through an output lens and concluded: The car needs improving:
Better fuel economy
Modern navigation
Better acceleration to get me there quicker
Better seating, comfort, air con, in built Haribo dispenser…
I get handed a new car for the journey, perfectly designed to suit the journey.
Would I have used it…?
Well, maybe not.
At that point, I wasn’t feeling confident. I didn’t know the journey, but I knew Molly. Every vibration, every noise, how it felt to drive.
Yes, a brand new big comfortable efficient car is better, but it didn’t have the thing that would have changed my behaviour. Familiarity.
This is the digital adoption problem, and what happens when you only look at outputs, and not outcomes.
You can build something that delivers, but you’ve not understood behaviours and change.
Evaluation done better
Good evaluation doesn’t start with ‘what we can count’, it starts with ‘what does change look like’.
That question leads you somewhere different. This is why Theory of Change helps. It maps how your activities lead to the outcomes you care about, and what assumptions and evidence lie beneath that.
This leads you to outcome measures, not just output measures.
You understand not just ‘how many people attended’ your event, but ‘what did they know, feel, or do differently’ because of it. This leads you to think about adoption and behaviour change.
Things worth planning for and measuring, because only then do you understand whether you’ve spent time and money doing things well.
My trip to France was all about learning to drive in another country. It was about drinking actual French wine in actual France with my actual friends, after having got there myself.
It was memories and confidence and feeling great with a sore back, not miles clocked.
How Impact Simplified can help
If any of this sounds familiar, or resonates with you, I can help.
I can work with you to:
Build a Theory of Change: map out the logic of how your work leads to change, making assumptions visible and identifying what to do better.
Develop an evaluation plan: Understanding your outcomes and what data you can use to measure it. Making sure you have the right measures in place from the start.
Present your evaluation story: It’s not just about gathering data, it’s doing something with it. And evaluation can be a story, just like my road trip with Molly. Your funders, senior leaders, and wider stakeholders want to know the story behind your work and what impact it has had, not dry numbers and figures.
Evaluation does not have to be complicated to be done well. But it does have to be thought through fully.
If you would like to talk about what this looks like for your organisation, get in touch.
