I recently had Professor Adrian Sargeant of The Philanthropy Centre on the line, so I asked a quick, but a very important question:
When should a fundraiser NOT bother with testing?
Really.
We go on and on about the importance of testing, and yet there are many times when testing is a complete waste of time, money, and energy. And it can teach us nothing at all … or worse yet, give us the wrong information.
Here’s when you should not test:
- When the number of donors you’re working with is too small to give you statistically significant results. When your number is too small, there’s more noise than signal. You stand a high chance of getting a false reading.
- When the notion you’re testing is already well established by other similar fundraisers.
- When it’s not clear what you’re trying to discover.
You’ll find Adrian clears the air on this difficult and complicated topic!
Find out even more about what it takes to raise money through the mail. Take the Moceanic online masterclass, 7 Steps To Creating Record-Smashing Direct Mail with Jeff Brooks. Click here to join The Fundraisingology Lab, and you can be taking the course within just a few minutes.
4 Comments. Leave new
Thanks, Sean and Adrian, for the cool video. Love the comment about TESTING could actually be about LEARNING!! Yippee!!!! Learning. Where have all the lifelong learners gone? Didn’t lifelong learning once used to be this really big goal? Hmmmm…..
You’re so right, Simone. A lot of testing is spent on “remedial learning” — that is, we test well-known best practices because someone in authority doesn’t believe things that are well known among fundraising professionals. In many cases this is necessary. But wouldn’t it be nice if the Skeptical Authorities would listen to professional advice (like I imagine they do to other professionals like plumbers and doctors), so we could use our testing to learn NEW things? The world is changing under our feet and we need to know a lot of stuff about new media, new behaviors, new demographics, and more … and that means testing things that we DON’T know!
I also like the important point about sufficient numbers for statistical significance. I had good experiences working in fundraising for-purpose organisations which had scientists on the staff, using the scientists for expert assistance on stats. Often helped to get quick, authoritative answers, and also the secondary benefit that it enlisted scientists in some understanding of fundraising. Even better if you have business analysts with business statistics background whose knowledge can be added to a fundraising team. Same secondary benefit too!
I love the double whammy benefit Roewen!
For those without access to such a good pool of experts, you can use Peter Schoewe’s useful online tool on Mal Warwick’s site here http://malwarwick.com/confidence-level-calculator. If you can’t get at 90% confidence (95% is better, but 90% gives you a good idea if not a fact) – that’ll make sense when you use it 🙂
sean