Measuring for Impact: It’s Not a ‘One-Size-Fits-All’

These days, it’s simply not enough to want to make a difference. More recently, donors are more and more interested in finding out how their contribution to a charity influences its overall success. And who could blame them? The nonprofit sector is growing at a rapid pace, with over 1.5 million nonprofit organizations to-date, according to the National Center for Charitable Statistics. It’s no wonder why donors are becoming interested in seeing what a nonprofit accomplishes with its proceeds. Donors feel good about themselves when they are a part of something impactful, substantial and significant, and they want hard evidence to show for it.

Donors want to see where their money is going; how their money is being spent; how much of it is actually being used for program expenses and not management or fundraising expenses (aka overhead); and how their contributions helped that organization achieve “success.”

This is where rating websites—or charity watchdogs—come into play. While there are a number of websites out there that provide information about nonprofits, a few of the big names out there are Better Business Bureau Wise Giving Alliance, Charity Navigator, Charity Watch and GuideStar. These sites are helpful resources for donors—and other nonprofits—to gauge how much impact that particular organization is making on society. But what do they mean for you?

Donors Need Clarity

What are donors looking for when they are quantifying how impactful an organization is? According to Jacob Harold, president and CEO of GuideStar, there are three important pieces of information that a donor looks for: the organization’s impact, the legitimacy of the organization and the organization’s finances.

“If there’s one word that I think donors should look for when looking at a nonprofit and making a judgment about its work, it’s “clarity.” Is that organization really clear about what its goal is; how does its work relate to that goal (what’s the logic behind their strategy); and are they making progress?” he said.

To meet donor expectations, nonprofits should focus on clarity. When an organization focuses on its clarity, it is putting its goal, strategies and measurements at the forefront. It’s important because with clarity, the donor has a clearer perception of the organization’s mission and overall goal.

“[Clarity] is about being clear about who you are and what you’re trying to do. It’s not always easy. Sometimes you have to take a conversation to your board or share it with your constituents to make sure that everyone agrees on the purpose of an organization. It seems like such a simple thing, but I think that’s the most important thing,” Harold said.

Challenges of Measuring Impact

It’s true—donors are curious and want to take the extra steps to measure for a nonprofit’s impact. Why is that? One reason may be that nonprofits are speaking more freely about the impact that their organization is making on the world, which is why they are advocating for it and why they are want and expect donations.

“We are now at a point where the nonprofit sector is this big and stable part of American society. With familiarity comes the desire to understand [questions like], ‘What is this all about, does it really matter and does it produce results that last?’ I think as the nonprofit sector matures, society has gotten used to us that their expectations have continued to grow,” Harold said. “I think our general expectations of having access to information are changing, and that is spilling over to philanthropy and the nonprofit sector.”

We also spoke to Michael Thatcher, CEO of Charity Navigator, who said, “I think we’ve kind of reached a terminal velocity where there’s been enough talk about where the money going, and now it’s more of what the money doing. The sector itself has been struggling with figuring out the right way to articulate impact and across all of the different areas of the sector.”

It’s a valid point. With all the information flurrying out there, how do nonprofits accurately measure their impact? How can organizations that focus on providing education to children quantify how much a child is learning? Or how about organizations that focus on feeding the poor? Are those individuals less hungry on a day-by-day basis? Specifically quantifying impact is not easy—it’s hard work. But because something is hard, doesn’t mean that we shouldn’t try. Harold mentions that the nonprofit sector needs to acknowledge that there are significant barriers to keep in mind when measuring for impact.

“When you have so many different kinds of organizations, it makes it harder to have comparable ratings of impact, but I think it’s a good thing because our diversity is a strength,” Harold said.

Thatcher points out that there is no existing common taxonomy or language for measuring impact across the different sectors, so it becomes cause-by-cause specific.

“Part of [the challenge] is just accessing information, so you’re not going to get impact information off of a publicly available tax form. That means there’s a challenge of access to the information,” he explained. “There are various attributes that you could look at whether it’s looking at constituent voice and raising the voice of beneficiaries.”

It’s almost impossible to pinpoint every variable to measure when looking at overall impact. The question becomes, “What kind of information is the donor actually looking for?” This is exactly what we asked Thatcher, and he believes the answer is donor-specific. What does the donor care about? Specifically—what kind of information can a nonprofit provide to the donor that will justify that their contribution will make a difference in society?

Robin Hood Foundation, a poverty-fighting nonprofit based in New York, developed an algorithm to measure the organization’s impact of its grants, which it calls its “metric approach.” According to its website, here is a play-by-play of that approach:

• Identify mission-related benefits. The organization determines every poverty-alleviating benefit generated by awarding a grant.
• Monetize outcomes. Robin Hood assigns a dollar value to each poverty-alleviating benefit.
• Calculate mission-related benefits. Robin Hood calculates the total dollar value of each benefit to those who live in New York.
• Interpret benefit/cost ratio. The organization assessed the impact of a grant by dividing its total benefit by total cost.

Read more on Robin Hood’s metric approach. Visit goo.gl/aY8hSf.

The Future of Impact Measurement

There is no one-size-fits-all strategy to measure for impact. There will never be a standardized method to measure an organization’s effectiveness. We believe it will always be an uphill battle, but one that will improve as technology advances.

Charity Navigator recently announced the launch of its Digitized Form 990 Decoder, an open-source project that explores nonprofit public tax records. The database provides over 1.7 million tax records—900,000 have already been processed.

“The Decoder is the start of a conversation,” David Borenstein, PhD, head of the Decoder project and lead data scientist at Charity Navigator, said in a press release from the organization. “It’s not complete by any means. In fact, we only mapped the data that we find useful for our work. By making this version of the Form 990 Decoder open-source code, we hope to get the ball rolling on a shared understanding of these data. Furthermore, we believe other developers will build off of what we’ve started to make a more robust and complete tool for the whole community to utilize.”

This development is a step in the right direction in terms of measuring for impact. With this tool, donors are now able to further their research on charities. In the past, unless you had experience in rigorous manual review or had super-savvy computer skills, it was exhausting to study these public records. Now, anyone (yes, anyone) can go to Charity Navigator’s website and use this technology.

The organization has provided detailed code, examples and instructions for using and extending the Decoder.

“In a sense, [the Decoder] is complete in the type of information Charity Navigator looks at from the 990. It is incomplete in that there’s a lot more information in the 990 that we don’t necessarily look at. What we are hoping is that the community of data scientists and related individuals who care about understanding what’,s in the 990 will continue to build and work on this tool, so we have a community tool for making sense of 990 data in a more structured way,” Thatcher said.

When looking into the horizon of impact measurement, Thatcher says that the future is bright. Charity Navigator is looking towards working with donors and charities to continue to improve and extend the evaluation methodology; to improve how the information if received by donors through rating websites; and to initiate a deeper and richer evaluation methodology.

“As we’re moving towards impact, Charity Navigator is not going to solve this on our own, [because] if it was a solvable problem by one organization, it would have been solved a long time ago. We’re really looking for more engagement and working with different cause areas to come up with the right things to measure and to do that as a joint effort. It’s a future that we are going to be much more
interactive with the sector to work towards what I believe to be common goals,” Thatcher said.

For GuideStar, The next step is investing more of its efforts into pushing out its platinum level data, which will help nonprofits share more details on their organization’s progress and will enable donors to learn more about the organization’s impact.

“There’s a bunch of exciting stuff that [GuideStar] is working on, but a lot of [our efforts are focused on] taking the core strategy we already have and making sure that reaches a larger scale,” Harold said.