Jennifer Lentfer, Director of Communications at IDEX, joined us for an overview of impact measurement for the fifth session of the Social Impact Academy. We discussed several assessment tools and methods that can make your organization more effective.
Session Recording: http://www.meetingburner.com/b/globalgiving/view_recording?c=HP7UJJ&h=f
Six ways to improve your organization’s credibility
- Know your audience
- Scale back the flowery language
- Back up your claims
- Cite your sources
- Show, don’t tell
- Be honest about what you know and what you don’t know
What makes evidence credible? Use the three classical modes of persuasion.
- Logos: Facts and figures, proof of a causal relationship
- Pathos: Emotional appeal, stories of impact have deep implications
- Ethos: Legitimacy and credibility, convince your audience to trust you and take action
Legitimize your stories with data and humanize your data through emotional response.
What can we expect along the way to social impact? Breakdown your roadmap.
- Problem analysis: What are we trying to change?
- Baseline: A clear understanding of the situation before the project began
- Goals, objectives: Where do we want to go?
- Strategy, activities: How are we going to get there? Ex. School feeding
- Output: Have the activities taken place? Ex. Children received more nutritious food
- Outcome: Did the change happen? Ex. Children’s weight and height increased
- Impact: Was there change at community/societal level? Ex. Healthier children
- Indicators and targets: How do we know we are on the right road?
Tools to Organize Impact Assessments
Road-mapping tools can help to define how you expect to reach your social impact goal. Below are two examples of road-mapping tools. Both provide visual approaches to help you organize the planning, implementation, monitoring and evaluation of projects.
- Logical Frameworks, or Logframes, break down the pathway to social impact in a chart. It defines your goals, activities, and subsequent actions.
- Theories of change are another way to visually track your pathway to social change. It focuses on defining long-term goals and mapping backward to meet preconditions. This method is useful because it shows that plan-making is not a linear process as the logframe might suggest.
Choosing measurement methods
Keep cost and complexity in mind as you develop your plan; the more legitimate your methods are, the more resources it will require. Below are a few examples of measurements which span from the lowest cost and complexity (observations), to the highest (randomized control trials).
- Observation and routine recordkeeping: This usually involves things you are already doing! It is useful for documentation of specific activities. Be sure to make your observations systematic.
- Key informant interviews: This research technique is more than a normal conversation. It involves in-depth interviews used to collect information from a range of people.
- Focus group discussions: These explore the motivations and perceptions of a homogeneous group. It is useful because there is a larger number of people offering insights and you can quickly collect complex qualitative information.
- Surveys: Be careful when administering surveys, many groups are over-surveyed. Surveys are useful to present statistically accurate data to make correct generalizations.
- Randomized control trials: This involves having a control group and an experimental group. It is especially valued because it eliminates bias. It is the most legitimate and credible measurement method but certainly not feasible for everyone.
Data collection tips
- Use indicators to mark the progress you are making. Example indicators include a percentage of the target population, an average, a score in an index, a ratio, and the presence/absence of a condition.
- Do not collect more data than you can analyze, plan how the data will be used so you don’t waste resources.
- Record keeping matters! Always date documents and files.
- Integrate your measurements in day-to-day work.
- Be a learning organization. Take data, internalize it and shift your organizational structure to learn from the past and plan for the future.
- Indicators should be comparable. Use baselines for this, i.e. “Before our project started, this was the situation. Now this is the situation.”
Q: Can you talk more about the differences between outcomes and impact?
Jennifer: Here’s an example: our activity might be running a children’s camp. At the top level we are looking to see whether change is happening for the entire community, whether the society is changing—this is the impact. There is something between the activity and impact, we call these outcomes. When children go home do they take their new knowledge and apply it to their lives? That is an outcome.
Alexis: In our first session Mari shared that she views an outcome as an effect that happens when a targeted community has participated in a program. She considers impact as societal or community change.
Q: Do you have any additional resources you would recommend?
Jennifer: I am happy to provide a problem tree exercise which can help with the problem analysis stage. In terms of collecting data, there are lots of cheap resources you can use now. You can create surveys for mobile devices. You always want to match what you want to know with your methodology, so each organization’s methods will be different.
Alexis: A lot of our partners manage their data through simple systems like excel. Many use CRMs, customer relationship managers. It will be useful to discuss this question at our next session.
This post was written by Kennan Howlett, Program Team Intern.