Leveraging AI Without Compromising Your Nonprofit’s Mission

In 2023, the National Eating Disorder Association (NEDA) discontinued its expensive and consistently overwhelmed hotline, replacing it with a chatbot named Tessa. The chatbot seemed like a financially sustainable alternative to a human-staffed hotline that could offer unlimited time to help more people.

However, the chatbot was taken down just five days later, after users went public with screenshots of their chats. These screenshots showed Tessa offering users suffering from eating disorders advice on how to lose weight.

We can learn a lot from NEDA’s experience with Tessa to avoid problems with artificial intelligence (AI) while taking advantage of its strengths.

AI Has Many Use Cases

Organizations can augment their powerful human talent using AI. It can increasingly help organizations fill skill and even staffing gaps by writing copy; creating original images; predicting donor interest and net worth; routing client calls, chats or emails; screening resumes; and writing code. With every release, we face a set of new capabilities that could help us scale our mission quickly.

But AI can also threaten privacy, compromise security, introduce bias, compromise quality, and, as in the case of Tessa, threaten an organization’s mission directly. In the long run, concerns abound about AI replacing human jobs or converting formerly high-paying jobs into low-skill, low-paying jobs. How should nonprofits approach AI in a time of rapid technological change? Can we use these tools to expand the good we are trying to do in the world, or are they just too dangerous?

It’s easy to look at the facts of the chatbot case and wonder how a nearly 40-year-old organization made such a substantial and consequential misstep of replacing human hotline workers with a chatbot, but a little more investigation renders some interesting lessons for nonprofit leaders.

According to the organization that released the chatbot, it had worked with their vendor to carefully design a rule-based chatbot, where each potential response was intentionally crafted by eating disorder experts. A rule-based chatbot can be stilted and sometimes frustrating to use, but it cannot start generating its own responses and, therefore, cannot go rogue and offer weight loss advice to eating disorder clients.

In this case, the chatbot vendor incorporated AI into its software, probably assuming it would be an upgrade for its clients. AI allowed the chatbot to generate its own responses, some of which might have been helpful to a general audience, but were seriously inappropriate in the context in which they were deployed.

Despite these facts being public, there was still a great deal of backlash against the organization that replaced its hotline with a chatbot. Discourse across the affected population largely agreed that the organization neglected to deeply understand or rigorously test the chatbot they released. As of this writing, about 70% of the text in NEDA’s Wikipedia page is dedicated to the Tessa fiasco and its impact on users with eating disorders and the human workers it replaced. They were held responsible for the quality of service that they provided and the impact of their decisions, regardless of their intent.

Recommendations to Consider Before Using AI

The NEDA example renders several recommendations for nonprofit decision-makers when it comes to technology deployment and use.

• Do not use software to replace humans for tasks that require empathy.
• Do not use software to replace humans for tasks for which high quality is central to your organization’s mission.
• Do not use software for high stakes tasks without rigorously and regularly testing it.
• Ensure that someone involved in new software adoption understands how AI works, where it gets its training data from, and how that relates to the context in which it will be used. This person should convey the software’s limitations to decision-makers and users and keep apprised of software changes through release notes and testing when needed.
• Talk to staff about how they currently use AI, how they would like to, and what they are concerned about. Consider that someone in their reporting chain is likely not the right person to ask them these questions.
• Recognize, and alert staff to, the fact that software can change without notice in ways that materially affect its operation.
• Design software and AI policies with an eye toward ethics and impact, including alignment with your organization’s values and goals. Consider the impact on the population served, your workers, and society.
• Regularly revisit organizational policies and personal practices around AI use to react to technology changes.

In conclusion, AI can make things efficient and effective, but organizations need to pay attention to the details of implementation and invest in testing, training, and policy. Always better safe, than sorry.

The preceding blog was provided by an individual unaffiliated with NonProfit PRO. The views expressed within do not directly reflect the thoughts or opinions of NonProfit PRO.