From my perspective as a practitioner trying to apply market systems thinking, there are important challenges in the way scale has been interpreted and in how that interpretation is applied in practice.

This was the topic of a recent paper I co-authored, developed by the Leveraging Economic Opportunities project. Like many challenges that evolve over a period of time, a series of perceptions and decisions has combined to foster donor environments that make it very hard to apply systems thinking effectively. These perceptions and decisions include the following: 

• Symptom indicators 
  Indicators ‒ especially the typical intermediate and impact indicators of most private-sector development projects, such as sales, uptake of specific practices or technologies, income, jobs, etc.‒ are symptoms of how a system has self-organised. Much like a fever or low blood pressure, disparity in income, high levels of poverty, tendency for zero-sum negotiations, high levels of corruption, and so on, are all symptoms of a social system that has self-organised in ways that caused those symptoms. Without context, these typical indicators provide very little understanding of the underlying causes of these symptoms, or if a project has successfully facilitated positive systemic change. 
• Targets are goals
  There are two concerns around targets as goals. First, by placing targets on symptom indicators and then making meeting those targets the main objective of the project, powerful incentives emerge to ignore underlying systemic constraints and to use project resources to mask symptoms. Similar to using a fever reducer without any concern about the underlying cause of the fever, once the temporary masking of the symptom wears off, the underlying cause almost always comes back. Second, setting targets implies a lack of dynamism in a system. You make the target or you do not. From a systems perspective, there is never a static state and what a project is trying to do is facilitate a change in the direction a system is moving – from a systems that is moving in an extractive direction to one that is moving in a more inclusive direction. For example, it is not only the income change that is important, but the way the income change came about that tells a systems thinker whether or not the change is real and durable. It is this combination of setting targets as goals and then interpreting a goal in a static way that makes systems thinking hard to apply. 
• Scale equals a large number
  In practice, when a project is asked if they have reached scale, they are being asked about how large a number have they been able to report for key indicators. When this concept of scale is applied to targets, the incentives intensify around getting higher and higher numbers. The pressure to increase the size of targets and achieve those targets creates a reinforcing feedback loop deepening the perception that symptoms are causes.

When this combination of issues is applied to an accountability regime that focuses on an individual project, the incentives are intensified to ignore underlying causes and double down on masking symptoms. Whether value for money or metrics related to assessing attribution in achieving a target, accountability linked to masking symptoms when focused on a single project creates multiple unintended consequences in how projects operate, including the following:

• Hierarchical management that devalues learning
  There is an implicit top-down framework that is applied when pressure to achieve numeric targets is asserted on an organisation. As the person at the top pushes staff below them to achieve specific numeric targets they are more and more likely to reward the staff that meet the targets even if those targets are achieved in ways that are unsustainable. Incentives mount for staff to push positive numbers up and suppress any learning related to underlying causes.
• Replication instead of real change
  Because a project is simply assessed based on numbers it achieved for its symptom indicators, there is substantial pressure to replicate interventions to produce more of the same thing. Systemic change cannot happen through replication of interventions that mask symptoms.
• Devaluing project-to-project cooperation
  As projects are stepping stones to follow-on and other work in a country, the pressure to achieve attributable targets mounts. This pressure around attribution at the individual project level creates strong incentives to work independently of other projects. Because projects are pushed to ensure they control the process of masking symptoms ‒ in order to demonstrate attribution ‒ the incentives align to limit cooperation.

Understanding these drivers of poor project performance is important for a couple of reasons. If we want to encourage faster and faster cycles of learning we need to rethink the way the current development environment catalyses incentives to not learn or adapt. A key aspect of the environment that needs further exploration is the idea of evidence. Evidence requires context, and from a systems practitioner perspective, much of what is defined as evidence is not really evidence as it does not provide insight or understanding. For example, when someone asks about evidence supporting systems thinking approaches I interpret the question as asking how well do system approaches mask symptoms ‒ since they want to know how large a number could the project report for a symptom indicator. The evidence I am keen to develop is around the affect the project had in changing key patterns and/or biases that relate to underlying causes. For example, in my current project, we are keen to understand the project’s effect on investment patterns, customer relationship patterns, and trust patterns in commercial relationships.    

I think what is needed to move the conversation forward is a complete rethink of the goal of projects, and a willingness to include more experimentation at the project level.  Assuming the goal of a development project is to catalyse systemic change, then an area that I am particularly interested in is the development of system health indicators or indicators that frame system inclusiveness, and this will be the topic of my next post. In the meantime, read the briefing paper, Reconsidering the concept of scale in market systems development.