Performance Measurement and Social Enterprise

Katie Hill, India Portfolio Manager at Acumen Fund, wrote the following post for Beyond Profit, as part of their series on track leaders in advance of  the Khemka Forum on Social Entrepreneurship, to be held in Hyderabad, India on December 8-9, 2009.

I recently visited rural Uttar Pradesh and met with a number of customers in off-grid villages who had purchased a D.Light Design solar lantern to replace their kerosene burner. D.Light strives to deliver both a financial and social return; Acumen Fund, my employer, invested in D.Light based on the hypothesis that this light will bring a higher standard of living—improved education from evening study hours, cost-saving from reducing kerosene expenditure, enhanced livelihood from adding more hours of productivity to a general store or tailor. Speaking with the customers in the village, the social impact seems obvious – literally in front of your eyes. But, aside from anecdotes, has D.Light or Acumen Fund “proven” that this impact exists?

Today, I sifted through the quarterly performance data for GEWP, a business selling low-cost drip irrigation products to small-holder farmers across India. Working for a social venture fund, my job involves not only tracking the financial viability of each company we invest in—monitoring revenues, profits, cash flows, and expected exit values—but the social value being generated. Even more, we hope to demonstrate the “social return on investment”—for every dollar we invest in GEWP, how many smallholder farmers double their incomes from precision agriculture? Taking this even further, how do I compare the impact of the GEWP drip unit to the impact of the D.Light solar lantern? These are only some of the many challenges social investors face.

The social entrepreneur’s challenges are different than those of the social investor. Any entrepreneur (social or otherwise) struggles with data collection when he or she has a small team, a limited budget and rudimentary systems. The social entrepreneur has similar issues which are compounded by the need to collect social impact data, not only to track her double bottom line and to answer to her stakeholders, but also as part of good business management. Consumer data is crucial to identifying growth opportunities or knowing where your business may be going astray. One must understand the customers’ demographics, what motivates them and what they value.

In this short post, I hope to share some of the impact measurement trends Acumen Fund has seen in the last decade, as well some thoughts on solutions to the toughest challenges.

Why Measure?

Why measure? It’s costly. It’s often muddled, and difficult to really “prove” anything. But I would argue that the future of the social enterprise sector—this idea (and it is still only an idea) that enterprises may be a more efficient means of improving the lives of the poor—rests entirely on our ability to get this right.

Social enterprise seems to be reaching a tipping point—new investors and enterprises are popping up everywhere, buzz words are flying. With excitement comes scrutiny. Social enterprises and investors had better be sure that they can answer tough questions:

• “Have your products/services actually reached the ‘poor’ (Related: How do we define poor?)?
• Are your investments drastically improving lives or only marginally?
• Is the business approach actually the most cost-effective means to reduce poverty or is this hype?”

You can take a little sigh of relief. A lot of extremely intelligent people have already put their brainpower towards these issues. I only have space to name a few.

• The “social return on investment (SROI) framework” has contributed a compelling theory for social investors.
• The essay “Measuring Innovation: Evaluation in the Field of Social Entrepreneurship” by Mark Kramer synthesizes a range of ideas around how to develop practical and balanced measures of impact.
• The recent framework, “The Base of the Pyramid Impact Assessment Framework: Enhancing Mutual Value Creation” by Ted London suggests how to account for the positive and negative impact of social ventures.
• The “Catalog of Approaches to Impact Measurement,” by SVT Group, offers a number of useful tools.

Clearing the biggest hurdles

The challenges of impact measurement may seem infinite. I am just going to name a few biggies.

1. Getting the data

The sheer logistical challenge of data collection is often overlooked. Bootstrapped social enterprises often don’t have big budgets or extra staff to complete wide customer surveys. So, we need to use what we’ve got. GEWP, for example, can use their 1-yr warranty cards to collect impact data. LifeSpring Hospital, a maternity hospital chain targeting low-income families, can use its registration process for new mothers to collect pertinent data, such as education backgrounds or household income levels.

From the investor seat, Acumen Fund experienced data scarcity challenges in our early days, and we’re still making improvements here. Simply collecting 5-7 meaningful data points every quarter, let alone the 30 we strive for today, at first seemed like a Herculean feat. Now, we get monthly data from our investees.

2. Inconsistency of metrics in the field

It seems like every new organization is reinventing the measurement wheel and, therefore, each investor is using slightly different metrics. One solution is IRIS. Evolving out of the Global Impact Investor Network (GIIN), a small group of stakeholders came together to think about a taxonomy of consistent metrics for social and environmental impact. The result has been Impact Reporting and Investment Standards (IRIS), which is an open source of common terms and indicators for financial, operational and social metrics on initiatives ranging from agriculture to education to healthcare. This common language is the first step towards truly building an industry.

3. Analyzing the data

There is no point in collecting numbers unless you’re learning something from them. So, how do we go about calculating meaningful indicators from a slew of data? It doesn’t have to be fancy. Use a pencil and paper, Google spreadsheets; use whatever works. As companies scale up, with high volumes of data and multiple locations, more sophisticated tools might be necessary. These could be standard MIS systems like Tally or Salesforce.

As investors, we faced similar challenges of moving from bulky spreadsheets to something more functional; we could not find an appropriate off-the-shelf software tool. So, we co-developed Pulse with Google.org. Pulse is a web-based platform for tracking and managing social investments. Pulse has already been used by a number of peer investors and is now being taken to the next level with the support of Skoll Foundation, Lodestar Foundation and Salesforce.com.

4. Correlation vs. causality

One of the trickiest points on impact measurement is whether we can ever have confidence that our initiative caused the intended outcome. For example, if patients from LifeSpring Hospital demonstrate improved health indicators in mothers and babies, is it because of their visits to LifeSpring (causation) or is it because of another aligned factor—the family cares about their health, so they both visit LifeSpring and engage in a number of other healthy activities, like drinking safe water (correlation). Randomized control trials, through players like MIT’s Poverty Action Lab, are the “gold standard” in proving causality.  In India, IFMR is also making great strides. These evaluations can be costly and are very time-intense. They should be used strategically and will require additional resources, as most social enterprises with thin margins can’t afford this line item.

5. Cost-effectiveness of measurement

To the point above, there is a general sentiment that social and environmental impact measurement is expensive and social enterprises don’t have an endless budget. It may be helpful here to distinguish between social outputs and social outcomes. For example, if SKS Microfinance distributes HUL’s Purit water filter to its borrowers, the “output” is X number of homes with access to clean drinking water. The “outcome” is the demonstrable impact—reduced incidence of water born diseases that results in healthcare cost-savings, higher school attendance, etc.

So, until the cost of doing these rigorous assessments fall, we think it is our responsibility to count the outputs as consistently as possible. The conclusions you can draw from these outputs may not be made with scientific rigor, but can inform businesslike decisions and raise important questions on impact.

Start Somewhere

We would argue that the lack of precision in social impact measurement is no excuse for not trying. Accurate impact measurement will only emerge if we start making imperfect calculations today and constantly improve upon them. There are a handful of impact measurement tools/methodologies out there. Each one in isolation may be problematic, but combined and cross-checked, the effort could lead to collective accuracy. And we need this collective accuracy if we hope to ever demonstrate that the social enterprise hypothesis holds water.

via blog.acumenfund.org

Posted via web from socialbizcoach's posterous