What Should You Test?

The question we as fundraising consultants are asked most often also is the question we keep asking ourselves: What elements of a direct-mail package are most important to test?

The question is important. What makes direct mail so successful as a fundraising medium is that we can track results and learn from them.

Everyone involved in direct mail makes decisions based on anecdotal information and on personal preferences. We also depend on experience: We compare one year’s results to another year’s results, or we line up returns from one package mailed in June against returns from another package mailed in September.

But those judgment calls are at best a calculated risk. At worst, delusions.

Testing is a must

To make consistent improvement in your fundraising program, you must test. That means mailing random samples (with test and control panels of names from your own donor list or from prospect lists) at exactly the same time and then subjecting those results to the rules of statistical validity.

But the catch is that even if you’re committed to testing, you must mail a large enough quantity to achieve statistical validity: To conduct reliable tests — with some important exceptions — your house list should be 10,000 names or larger, and an acquisition effort (prospect mailing) should include at least 50,000 pieces.

Tests to try

If that’s the case, then here are three of my favorite elements to test:

1. In fundraising mail, the test that usually makes the greatest difference involves the amount you request. That’s particularly the case in acquisition mailings: Will you persuade more individuals to become members or donors by asking for an initial gift of $15, $20, $25 or $35? For some organizations, asking for a lower initial gift results in a higher response rate without a decline in the average gift.