When Is Testing NOT Worthwhile in Fundraising?

VIDEO: Sean Asks Adrian Sargeant: When Is Testing NOT Worthwhile in Fundraising?


I recently had Professor Adrian Sargeant of The Philanthropy Centre on the line, so I asked a quick, but a very important question:

When should a fundraiser NOT bother with testing?


We go on and on about the importance of testing, and yet there are many times when testing is a complete waste of time, money, and energy. And it can teach us nothing at all … or worse yet, give us the wrong information.

Here’s when you should not test:

  • When the number of donors you’re working with is too small to give you statistically significant results. When your number is too small, there’s more noise than signal. You stand a high chance of getting a false reading.
  • When the notion you’re testing is already well established by other similar fundraisers.
  • When it’s not clear what you’re trying to discover.

You’ll find Adrian clears the air on this difficult and complicated topic!

Find out even more about what it takes to raise money through the mail. Take the Moceanic online masterclass, 7 Steps To Creating Record-Smashing Direct Mail with Jeff Brooks. Click here to join The Fundraisingology Lab, and you can be taking the course within just a few minutes.


  • Sean Triner

    Sean Triner is a Co-Founder of Moceanic and Pareto Group. With over three decades of experience in fundraising and a Bachelor of Science (Honours) in mathematics, Sean has a wealth of fundraising expertise to share with Moceanic members and blog readers. He is also a coach and available for bookings on the Coaching+ program for fundraisers to help answer your questions and work hands-on directly with you.

    View all posts
Previous Post
How to Bulk Up Response to Any Fundraising Ask with Matching Funds
Next Post
How Much Is Too Much for High-End Direct Mail Packs?

Related Posts

4 Comments. Leave new

  • Thanks, Sean and Adrian, for the cool video. Love the comment about TESTING could actually be about LEARNING!! Yippee!!!! Learning. Where have all the lifelong learners gone? Didn’t lifelong learning once used to be this really big goal? Hmmmm…..

    • You’re so right, Simone. A lot of testing is spent on “remedial learning” — that is, we test well-known best practices because someone in authority doesn’t believe things that are well known among fundraising professionals. In many cases this is necessary. But wouldn’t it be nice if the Skeptical Authorities would listen to professional advice (like I imagine they do to other professionals like plumbers and doctors), so we could use our testing to learn NEW things? The world is changing under our feet and we need to know a lot of stuff about new media, new behaviors, new demographics, and more … and that means testing things that we DON’T know!

  • I also like the important point about sufficient numbers for statistical significance. I had good experiences working in fundraising for-purpose organisations which had scientists on the staff, using the scientists for expert assistance on stats. Often helped to get quick, authoritative answers, and also the secondary benefit that it enlisted scientists in some understanding of fundraising. Even better if you have business analysts with business statistics background whose knowledge can be added to a fundraising team. Same secondary benefit too!

    • I love the double whammy benefit Roewen!
      For those without access to such a good pool of experts, you can use Peter Schoewe’s useful online tool on Mal Warwick’s site here http://malwarwick.com/confidence-level-calculator. If you can’t get at 90% confidence (95% is better, but 90% gives you a good idea if not a fact) – that’ll make sense when you use it 🙂


Leave a Reply

Your email address will not be published. Required fields are marked *

Fill out this field
Fill out this field
Please enter a valid email address.