Does weighing the pig change the actual pig?  Evaluating the impact of school partnerships

In education we often talk about “weighing the pig” as a metaphor for assessing student progress. “Does weighing the pig make it any fatter?”, we ask.   Academic monitoring in schools has become the engine of accountability and SLTs up and down the land are forever discussing whether to spend more time learning or testing, in other words, feeding or weighing said pig.

What about weighing the partnerships pig?  How do we go about measuring the effectiveness of educational partnerships?  Do they make a difference?  Enough to warrant the time and money spent on them?

ImpactEd are experts on measuring the impact of educational interventions. They don’t shy away from the challenge of evaluating of complex partnership projects which involve multiple subjects, years and even school types.  I was delighted to team up with them last month to deliver a brace of webinars on “Partnerships with Impact” – Joe McGinn from ImpactEd as the expert and me as the school-experienced practitioner!

Joe’s advice is to keep it simple: ensure your impact evaluation is proportionate to the data you want to collect and the reason you are collecting it.  Don’t weigh the pig more than you have to, or go to such lengths that you don’t have capacity to then analyse all the data you have collected.   Have a clear aim and ask the right questions to see if you’ve fulfilled it.  But maybe, I ventured, simply knowing that we’re going to have to get the weighing scales out will change the actual pig we select for measurement?  Might we make decisions about what partnership projects to pursue based on how easy they will be to assess for impact?

Sadly, we know this is a real danger in education.  I’ve been talking about this with my friend Cathy Hills, a PhD student who is investigating data collection in schools.  She sees schools grappling with the balance between the time spent fattening or weighing the pig, but also sees data driving more than just assessment timetables.  Even curriculum design can be motivated by data; the decline of GCSE music in schools, for example, is seen as a consequence of the EBacc and Progress 8 metrics.  This might have been unintended but was surely foreseeable.  

How could this play out in partnerships?  Let me illustrate.  If the aim of a partnership is to club together to buy in external physics CPD, for example, the impact on the teachers involved is fairly straightforward to assess.  But what about getting students together for a mindfulness workshop, or an outward-bound day designed to build confidence?  The physics CPD pig is more easily weighed than the rather more slippery student confidence one, so is that likely to influence our choice of activity?  After all, “Not everything that counts can be counted, and not everything that can be counted counts” – famously quoted by Einstein but probably correctly attributed to William Bruce Cameron.

In fact, as Joe showed the delegates at our webinars, there are myriad metrics out there which have been researched and shown to be effective in measuring all sorts of social and emotional skills, such as grit, self-efficacy and sense of community.  But they may not be widely known about – after all, most partnership leaders in schools are teachers, not social scientists.  However, ImpactEd can help, and in the meantime, here are my suggestions for embarking on weighing your partnerships pig:

• plan your partnerships strategy by addressing the need you are trying to meet and having a clear aim.
• think about what the outcomes might be (benefitting all partners) and how you’d measure them.
• start small, with a pilot perhaps, and tweak your plans and your measurements according to what you find.

Good luck with your impact evaluation; I’d love to hear your take on what works well.

_{[Featured image picture credit: Photo by Diego San on Unsplash]}