4 Keys to Designing Better Surveys
If you want to prove a point, nothing is better than a survey. You can, as you likely know, get any result you want by positioning the questions carefully. Today, it’s easy for anyone to put together and send out a survey using tools like SurveyMonkey. But surveys can be difficult to construct and, unless done so thoughtfully, can provide you with misleading information that will take you down the wrong path.
Surveys that measure people’s attitudes are the tricky ones. Getting people to tell you what they feel, or why they did or would do something is fraught with peril.
The ‘Willing-and-Able’ Problem
When reporting any attitude, psychologists talk about the “willing-and-able” problem. Many times, people aren’t willing to give you accurate information. For example, they may want to portray themselves as someone who is actually more socially active than they actually are.
On the other hand, people are often unable to tell you what led them to do certain things since much of what we do are the result of unconscious motivations.
The Theory of Planned Behavior
Compounding the difficulty getting accurate information is the fact that many attitude measures are weak at predicting actual behavior. The theory of planned behavior was developed in the 1980s by psychologist Icek Ajzen, and it aims to explain the link between attitudes and behavior. Studies show that this theory is good at predicting what people will actually do. It is based around three elements that combine to create a person’s intent to act in a particular way:
- The person’s attitude towards a specific behavior (including how often they have done it in the past).
- What they see as the normal attitudes of others towards that behavior.
- Things that may limit their own behavior.
For example, if we take the behavior of volunteering to raise money for a walk, we might ask the person whether they think it was worthwhile, how often they have done it before (their own attitude), whether they think their friends and family would like to volunteer (the normal behaviors of their peer group) and if it is something they have the time to do (things that might constrain their behavior).
We may hold a positive attitude towards a nonprofit’s mission, but if we are surrounded by people who don’t think it is important, or it is not feasible for us to do, we probably won’t volunteer.
The theory of planned behavior provides us with some practical ideas to improve survey design. For example:
1) Instead of asking how much the person likes your nonprofit’s mission, ask them how often they might volunteer.
2) Ask what might prevent them from volunteering.
3) Ask them how often they would have opportunities to volunteer.
4) Ask what they believe other people like them think of volunteering for your nonprofit, or whether their friends and family would approve of them doing so.
It can be easier for people to report how often they have performed a behavior on your behalf than to describe how they feel about it. Don’t push too hard on questions that ask people things that they have to guess at, like what they would do in the future. These are usually questions asking them to explain their motivations for doing something.
If you do ask questions about motivation, know that people might unwittingly fabricate plausible-sounding, but incorrect explanations. Your survey taker may not have introspective access as to why they really performed the behavior about which you are asking.
And all that means that you could be building a budget, event series plan or growth plan on highly suspect information.
Otis Fulton, Ph.D., spent most of his career in the education industry, working at the psychometric research and development firm MetaMetrics Inc., Pearson Education and others. Since 2013, he has focused on the nonprofit sector, applying psychology to fundraising and donor behavior at Turnkey. He is the co-author of the 2017 book, ”Dollar Dash: The Behavioral Economics of Peer-to-Peer Fundraising,” and the 2023 book, "Social Fundraising: Mining the New Peer-to-Peer Landscape," and is a frequent speaker at national nonprofit conferences. With Katrina VanHuss, he co-authors a blog at NonProfit PRO, “Peeling the Onion,” on the intersection of psychology and philanthropy.
Otis is a much sought-after copywriter for nonprofit fundraising messages. He has written campaigns for UNICEF, St. Jude’s Children’s Research Hospital, March of Dimes, Susan G. Komen, the USO and dozens of other organizations. He has a Ph.D. in social psychology from Virginia Commonwealth University and a Bachelor of Arts from the University of Virginia, where he also played on UVA’s first ACC champion basketball team.
Katrina VanHuss has helped national nonprofits raise funds and friends since 1989 when she founded Turnkey. Her client’s successes and her dedication to research have made her a sought-after speaker, presenting at national conferences for Blackbaud, Peer to Peer Professional Forum, Nonprofit PRO, The Need Help Foundation and her clients’ national meetings. The firm’s work is underpinned by the study and application of behavioral economics and social psychology. Turnkey provides project engagements, coaching, counsel and staffing to nonprofits seeking to improve revenue or create new revenue. Her work extends into organizational alignment efforts and executive coaching.
Katrina regularly shares her wit and business experiences on her and Otis Fulton's NonProfit PRO blog “Peeling the Onion.” She and Otis are also co-authors of the books, "Dollar Dash: The Behavioral Economics of Peer-to-Peer Fundraising" and "Social Fundraising: Mining the New Peer-to-Peer Landscape." When not writing or researching, Katrina likes to make things — furniture from reclaimed wood, new gardens, food with no recipe. Katrina’s favorite Saturday is spent cleaning out the garage, mowing the grass, making something new, all while listening to loud music by now-deceased black women, throwing in a few sets on the weight bench off and on, then collapsing on the couch with her husband Otis to gang-watch new Netflix series whilst drinking sauvignon blanc.
Katrina grew up on a Virginia beef cattle and tobacco farm with her three brothers. She is accordingly skilled in hand to hand combat and witty repartee — skills gained at the expense of her brothers. Katrina’s claim to fame is having made it to the “American Gladiator” Richmond competition as a finalist in her late 20s, progressing in the competition until a strangely large blonde woman knocked her off a pedestal with an oversized pain-inducing Q-tip. Katrina’s mantra for life is “Be nice. Do good. Embrace embarrassment.” Clearly she’s got No. 3 down.