Archive

Tag Archives: logic model

by Floyd Rumohr

Imagine a world without valid judgment: doctors unable to evaluate x-rays. Drinking water deemed safe just by looking at it. Judges drawing conclusions based on how they feel about a case rather than from standards and laws. Sounds like chaos.

Disorder at nonprofits isn’t always imaginary. It can be embedded in one organization’s culture and be a momentary lapse in judgment at another. Regardless of how often it appears, chaos is almost always certain to ensue if decision-making lacks valid reasoning and credibility, which is the degree to which judgments and conclusions are trusted and believed by others. Credibility is undermined when nonprofits make unsubstantiated claims about their work or if outputs alone are considered “impact.” The number of tickets sold at a theater performance is an output, for example, while the degree to which the audience is affected by that same performance is the impact. 

Sean Stannard-Stockton defines outcomes as “observed effects of the outputs on the beneficiaries of the nonprofit.” Impact is “the degree to which the outcomes observed are attributable to its activities.” Knowing how inputs and activities are causally connected to organizational impact is essential if nonprofits want to understand what’s working and what’s not, improve performance, and back-up claims. That’s where evaluation comes in.

Two basic kinds of evaluation are described by Dr. Jens J. Hansen of the Woodhill Park Research Retreat:

  • Formative evaluation is a kind of “stock-taking.” It can and should be done at regular intervals and provides vital information from which midcourse corrections will be made. Assessment and research play a big role here. Teachers often use formative evaluation and assessment methods to monitor how well students are learning and to make changes to the instructional methods and content depending on how those assessments come out. Focus is on the process.
  • Summative evaluation is the “final test” and demonstrates what has or has not been achieved in the strategic plan. “Here evaluation rigorously interrogates organizational performance with respect to whatever goals and objectives are nominated in the strategic plan.” A standardized reading test for fourth grade students is an example of summative evaluation. Focus is on outcomes.

Both approaches play a role in strategic planning because one provides information during which course corrections at the program level are made (formative) and the other generates credible information for broader organizational decisions (summative). Both reflect evidence-mining processes that characterize learning cultures and help to connect an organization’s inputs (causes) to its outcomes/impact (effect) — otherwise known as the logic model.

The logic model is not about simple math. You can’t “prove” it no matter how sensible it is. “The term ‘scientific proof’ is a misnomer: proof is a concept germane to mathematics, not science…Scientists, including social scientists, develop hypotheses about how the world works and then gather evidence to support or undermine those hypotheses. Whereas proof is black-and-white, evidence has shades of gray: it can be strong or weak, circumstantial or conclusive” (Moss).

Put more simply, there are degrees of evidence for claims about impact but not proof of those claims. For a small nonprofit, this process of evidence gathering is a significant investment especially if outside researchers are involved. Research grants from funders can be so highly competitive as to put them out of reach. So how does a small nonprofit with limited resources credibly gather such evidence?

Start with the organization’s logic model. What are the inputs and activities? What are the outcomes and impact of those activities? How do you know the impact is a result of organizational activities? If the organization has several outcomes described in its impact statement, it’s probable only one of them can be tackled at a time. Don’t try to do everything at once. Credible evidence in support of one outcome or strategic goal can be a good road to take when resources are limited. Independent researchers and evaluators are a helpful “outside eye” and more financially accessible to small nonprofits when evaluative activities are focused.

Like it or not, persuasive impact statements are here to stay. Emerging philanthropists with voracious data appetites will be looking more and more for credibility behind claims in public presentations on Guidestar, in grant proposals, and web sites.

If your organization relies more on fiction than on fact to describe its impact, it might be time for some research, evaluation, or assessment.

Next up: alliances and partners.

References and recommended reading:

  • A Strategic Planning Template for Dummies” by Dr. Jens J. Hansen, Woodhill Park Research Retreat: a helpful guide for nonprofit organizations with a comprehensible summation of evaluation in the context of strategic planning.
  • In Defense of Logic Models” by Ian David Moss, Huffpost Arts & Culture Blog: unpacks logic models and explains theory of change/program theory from arts organizations’ perspectives.
  • Getting Results: Outputs, Outcomes and Impact,” Stanford Social Innovation Review by Sean Stannard-Stockton.
  • Charting Impact: BBB Wise Giving AllianceGuideStar USA, and Independent Sector developed Charting Impact as a common presentation that allows staff, boards, stakeholders, donors, volunteers, and others to work with and learn from each other.
  • Stages of Learning Research Study” by Jane Remer and Floyd Rumohr: this four-year study at PS 145, Brooklyn, postulated research questions that provided an evidentiary basis for subsequent organization-wide summative evaluation.

by Floyd Rumohr

Mr. Spock is Star Trek’s indelible archetype of logic on the Starship Enterprise.

Logic is an organization’s description of the cause and effect relationship between what goes into a nonprofit program and what comes out of it. 

Logical thinking is always important but especially so during periods of downturn, recession, and shrinking resources because, without it, otherwise well-intended organizations may deviate from mission and purpose with disastrous results.

Picture it: a largely but not entirely hypothetical nonprofit literacy organization, which I’ll call “ORG,” was founded to fund other charitable groups but loses its primary source of revenue from a single funder after about fifteen years of operation.

ORG scrambles to find new sources of revenue by, among other things, contracting with schools to become a competitor with organizations it previously funded. While ORG isn’t new to the field, it is new to direct services and in this way has cannibalized the organizations it helped build. Sad to say, but this “eating of its young” is an “outcome” or “impact” of its activities.

There’s not much logic in that. Were logic involved, ORG might not have pursued the course it did. By severely reducing grant making activities, establishing new programs, increasing the size of its development department, and creating new accounts receivable systems and staff to manage earned income, the number of children who successfully completed literacy programming went from 14,000 to 4,000.

Decreased numbers of children learning to read and write are just the tip of the iceberg to ORG’s ferociously flawed logic. Some previously funded exemplary organizations did not survive withdrawal of its funding and dissolved. Others sputtered along and were forced to make dramatic cuts to programming. It’s no surprise that fewer children received help.

So why did ORG do what it did? There are many plausible answers not the least of which is a desire to sustain a worthwhile organization. Life cycles can be hard to accept for us humans who form emotional attachments to organizations and programs that have outlived their effectiveness.

What, then, would have been a more logical course for ORG to take? Perhaps it could have become a leaner technical assistance organization to the nonprofits it once supported if it aimed to sustain itself. If not, a graceful dissolution could have been rolled out. In this case, multi-year grants could have been provided to the agencies it once supported by exhausting its remaining endowment. Either mission would have been a more logical alternative given ORG’s primary purpose.

Where was ORG’s logic flawed? An organization’s logic model connects the time, money, effort and other resources spent on its programs to its outputs, outcomes, and impact. ORG’s theory of change, which according to Ian David Moss “‘unpacks’ the processes and factors that lead to successful outcomes exposing relationships between isolated variables that can then become the subject of research or evaluation” might explain what happened. ORG suffered less from fewer children served and more on inappropriate building blocks, such as creating an entirely new competing program, to achieve that long term goal.

A healthy logic model is one in which the allocation of time and money supports outcomes/impact that build competitive advantage. An unhealthy cycle is one in which resources are spent on activities that ultimately diminish impact. While ORG was able to sustain itself, for example, it did so at the expense of the field it once supported and had less impact on the very children it was founded to help.

In your organization or the groups with which you work: is there a high correlation between how it spins its input wheels and superior impact? Is the organization’s mission directly linked to observable outcomes? If so, have at it! If not, it might be time to diagram the cause and effect relationships between the inputs, activities, and outcomes. An equally important question: How do you know inputs and activities are causally connected to impact? 

Next up: evaluation.

References and recommended reading: