Fundraisers: Watch Your Backs
Let's say you punched me in the nose. (Don't worry about why you did it. For the purposes of this allegory, the reason doesn't matter.)
Immediately after you sock me, you go into a defensive pose. You crouch down and put up your guard, and prepare for me to respond in kind. That's the natural, normal response. You punch someone, of course that person is going to punch you back.
But what if I don't do the "normal" thing? From your perspective as the puncher, it might seem a very odd response. But my perspective as the punchee might be quite different:
- Maybe I was raised by Quakers.
- Maybe you're twice my size.
- Maybe I deserved it.
There are any number of perfectly normal reasons you don't get what seems to you to be the normal, rational response.
For fundraisers, making normal assumptions can be a costly mistake. It happens all the time in acquisition testing.
Let's say you mail your control package and a couple of well-thought-out strategic tests. And let's say one of the tests wins. This is great. You have a new control. You invested all that time and money, and risked something new, and it paid off.
But did it really? Actually there could be any number of reasons your test beat the control — that time.
- Maybe there were some anomalous issues with the list.
- Maybe your quantity was too small to be statistically reliable.
- Maybe it was just one of those things.
Accepting test results from a new package makes plenty of logical sense. But it's also rife with peril. It's just not possible to think of every scenario that could have made that package yield false positive results. And rolling out a package like that can lead to big headaches.
The solution, though, is simple: Back testing.
You back test by mailing your winning package a second time, in order to confirm its results. Unfortunately, back testing is not talked about, or done, nearly enough. After all, people reason, running the same test is expensive and time-consuming. And the test did prove itself after all.
It's perfectly normal to want to save all that time and money just to prove something that already seems obvious. But the sad fact is that the direct-mail highway is littered with packages that failed to live up to their initial test results.
And in the long run — after new rounds of testing and more months of waiting and analyzing results — those packages ended up costing a lot more time and money than the back test would have.
Willis Turner believes great writing has the power to change minds, save lives, and make people want to dance and sing. Willis is the creative director at Huntsinger & Jeffer. He worked as a lead writer and creative director in the traditional advertising world for more than 15 years before making the switch to fundraising 20 years ago. In his work with nonprofit organizations and associations, he has written thousands of appeals, renewals and acquisition communications for every medium. He creates direct-response campaigns, and collateral communications materials that get attention, tell powerful stories and persuade people to take action or make a donation.