How to go from drab to fab with evaluation

Last week I talked about how awesome program evaluation is, and how it helps you to quantify just how much your work changes lives. This week I want to go into more detail about that.

Many of you out there have big, beautiful goals for your programs—increase access to healthcare for low-income families, protect endangered species, advocate for human rights, provide high-quality education to all, etc. You sink time, effort, blood, sweat, and tears into that goal. You want to make a difference, and to make the world a better place. Evaluation can help you prove that you did that, and make your impact statements something that will really blow people away (in a good way).

An impact statement is essentially what you say in situations like this:

• When your spouse/parent/family member asks, “What do you do?”
• When your funder asks, “What were you able to accomplish with my money?”
• When a prospective donor asks, “Why should I fund YOU specifically?”
• When your (tired and overworked) team asks, “Is all this work really changing lives?”

Responses to these questions—aka impact statements—are only as powerful as the evaluation that goes into them.

Here’s an example:

Say that your program is a summer camp designed to spark interest in science, math, and engineering (STEM) in middle school girls. You’ve targeted middle school girls because this is typically the time in their life where girls drop out of the sciences, and this is why we end up with a seriously gendered science field. (And a seriously gendered science field is a serious problem for oh so many reasons.) Your program is thoughtful, and includes fun, hands-on education geared towards their interests, conveyed in language that resonates with them.
(Side note: This camp really existed for many years at Pacific University in Oregon, and I was fortunate enough to help evaluate it years ago. It is no longer active, but I’m sure there are many similarly awesome camps like it across the country… at least I hope there are!)

At the end of the camp, what can you say about what you’ve accomplished?

Impact statement without evaluation
Here’s what you can say if you weren’t able to do any evaluation:

“We ran a three-week summer science camp for middle school girls. We exposed them to a variety of content—biology, chemistry, physics, etc. I think it really made a difference, and that as a result of going through our camp, these girls are more interested in science and more likely to become scientists.”

Hmmm, okay, that’s pretty interesting… but not great. It’s all about what you did, not what those activities did for the campers or for science in general.

Impact statement with some evaluation
Now let’s say you did a little evaluation—a brief survey of the campers at the end of the camp—here’s what you can add to the statement above:

“95% of the campers learned something new at camp. 93% plan to register in a high-level science class when they get to high school in the fall. 76% have career aspirations that relate to science.”

WOW! Now that’s impressive. Now we know what you did, and what the impact was on the campers. They learned things, they’re planning to change their behaviors, and it looks likely that they will go on to reach your end goal—going into science and evening out the gender discrepancy in this incredibly important field.

Impact statement with great evaluation:
What if you do a really stellar, extensive evaluation? Say it’s something like pre-and-post tests for the girls, and regular follow-ups for the next few years to track their progress. If so, here’s what you can add to your impact statement:

“99% of our campers graduated from high school with above a 3.0 GPA. 85% of them enrolled in college after that, and 75% of those majored in a science field. This means we helped add 75 talented young women to the sciences who are now ready to join the workforce and help even out the gender issues. We are changing the face of science with our work!”

Amazing!  Upon hearing that, I am really and truly impressed that not only are your efforts making a difference, they’re reaching your goals.

Bottom line:

With evaluation, you are able to provide compelling, evidence-based answers to questions like, “What do you do?”, or “What were you able to accomplish with my money?” or “Why should I fund your program?”. You can definitively demonstrate that your activities are changing lives, and the future of science as a whole.

Now that is a compelling impact statement.