4 Keys to Designing Better Surveys
If you want to prove a point, nothing is better than a survey. You can, as you likely know, get any result you want by positioning the questions carefully. Today, it’s easy for anyone to put together and send out a survey using tools like SurveyMonkey. But surveys can be difficult to construct and, unless done so thoughtfully, can provide you with misleading information that will take you down the wrong path.
Surveys that measure people’s attitudes are the tricky ones. Getting people to tell you what they feel, or why they did or would do something is fraught with peril.
The ‘Willing-and-Able’ Problem
When reporting any attitude, psychologists talk about the “willing-and-able” problem. Many times, people aren’t willing to give you accurate information. For example, they may want to portray themselves as someone who is actually more socially active than they actually are.
On the other hand, people are often unable to tell you what led them to do certain things since much of what we do are the result of unconscious motivations.
The Theory of Planned Behavior
Compounding the difficulty getting accurate information is the fact that many attitude measures are weak at predicting actual behavior. The theory of planned behavior was developed in the 1980s by psychologist Icek Ajzen, and it aims to explain the link between attitudes and behavior. Studies show that this theory is good at predicting what people will actually do. It is based around three elements that combine to create a person’s intent to act in a particular way:
- The person’s attitude towards a specific behavior (including how often they have done it in the past).
- What they see as the normal attitudes of others towards that behavior.
- Things that may limit their own behavior.
For example, if we take the behavior of volunteering to raise money for a walk, we might ask the person whether they think it was worthwhile, how often they have done it before (their own attitude), whether they think their friends and family would like to volunteer (the normal behaviors of their peer group) and if it is something they have the time to do (things that might constrain their behavior).
We may hold a positive attitude towards a nonprofit’s mission, but if we are surrounded by people who don’t think it is important, or it is not feasible for us to do, we probably won’t volunteer.
The theory of planned behavior provides us with some practical ideas to improve survey design. For example:
1) Instead of asking how much the person likes your nonprofit’s mission, ask them how often they might volunteer.
2) Ask what might prevent them from volunteering.
3) Ask them how often they would have opportunities to volunteer.
4) Ask what they believe other people like them think of volunteering for your nonprofit, or whether their friends and family would approve of them doing so.
It can be easier for people to report how often they have performed a behavior on your behalf than to describe how they feel about it. Don’t push too hard on questions that ask people things that they have to guess at, like what they would do in the future. These are usually questions asking them to explain their motivations for doing something.
If you do ask questions about motivation, know that people might unwittingly fabricate plausible-sounding, but incorrect explanations. Your survey taker may not have introspective access as to why they really performed the behavior about which you are asking.
And all that means that you could be building a budget, event series plan or growth plan on highly suspect information.
Katrina VanHuss and Otis Fulton have written a new book, Dollar Dash, on the psychology of peer-to-peer fundraising. Click here to download the first chapter, courtesy of NonProfit PRO!
Katrina VanHuss is the CEO of Turnkey, a U.S.-based strategy and execution firm for nonprofit fundraising campaigns. Katrina has been instilling passion in volunteer fundraisers since 1989 when she founded the company. Turnkey’s clients include most of the top thirty U.S. peer-to-peer campaigns — Susan G. Komen, the Cystic Fibrosis Foundation, the ALS Association, the Leukemia & Lymphoma Society, as well as some international organizations, like UNICEF.
Otis Fulton is a psychologist who joined Turnkey in 2013 as its consumer behavior expert. He works with clients to apply psychological principles to fundraising. He is a much-sought-after copywriter for nonprofit messaging. He has written campaigns for St. Jude’s Children’s Research Hospital, The March of Dimes, the USO and dozens of other organizations.
Now as a married couple, Katrina and Otis almost never stop talking about fundraising, volunteerism, and human decision-making – much to the chagrin of most dinner companions.
Katrina and Otis present regularly at clients’ national conferences, as well as at BBCon, NonProfit Pro P2P, Peer to Peer Forum, and others. They write a weekly column for NonProfit PRO and are the co-authors of the 2017 book, "Dollar Dash: The Behavioral Economics of Peer-to-Peer Fundraising." They live in Richmond, Virginia, USA.