Selecting the appropriate model for diminishing returns

In a previous post, Max suggested two key ways of thinking about diminishing returns: funding gaps and returns functions. He also set out two classes of considerations that are generally desirable in models: closeness of fit and clarity.

In this post, we set out some considerations when choosing between funding gaps and returns functions. An underlying theme is that returns do not diminish so steeply as people often think, or funding gap models suggest. Thinking in terms of returns function models is normally a clearer model of true returns, but funding gap models can be useful for some purposes.

One desiderata for a good model is that it closely approximate true returns. To assess which models do this well, it’s useful to consider what true returns might look like. In particular, it’s important to consider whether returns diminish steeply beyond any particular point, because this is a point of difference between funding gap models and returns functions.

First, assume that organizations tend to take their highest-impact opportunity first, and then take the next best opportunity, etc. (This is the main theoretical reason why we expect diminishing returns in the first place). Now suppose that the value of different opportunities is relatively continuously distributed: there aren’t big gaps where some opportunities are much better than the next. If this is the case, then we would expect the returns to additional funding to decrease smoothly rather than discontinuously, as the robust relative funding gap model suggests.

Second, even if returns do diminish steeply beyond some point, we may be uncertain where that point is. For instance, I may be certain that AMF can only execute a certain number of new distributions this year. However, I’m unsure what that number is, and I’m unsure how big each distribution will be, so although I’m sure there is some hard upper limit beyond which donations will be useless, I don’t know what that upper limit will be. (Note that this seems to be roughly the model used by GiveWell.) To account for this uncertainty, I should focus on the value that I expect my donations will generate. Assuming that my uncertainty about the value of the upper bound is smooth, then in expectation my estimate of the returns to additional funds declines smoothly, even if I think that funds will be useless beyond some funding level.

Third, we might expect that even when an organization can’t spend more on its core programs, returns will diminish smoothly. Suppose that past a certain level, an organisation is unable to use funding on its core programmes. Additional funding has two significant benefits to the organisation, which push against returns diminishing steeply.

Reserves: The organisation can hold additional funding in reserve, to spend on core programmes in subsequent years. Assuming that core programmes are roughly as effective next year, additional funding mostly reduces the funding needs of the organisation next year, thereby freeing up money for those donors who would have given next year. Assuming those donors still donate that money somewhere else, then their alternate donations are likely to produce at least almost as great value as this organisations’ core programmes. An additional benefit of increasing reserves in this way is that this may allow the organisation to spend less time on fundraising (either by putting less time into prospecting funding, or by delaying funding rounds). Since fundraising is often carried out by relatively senior, valuable staff, this effect may be significant.

Capacity: Even if funds cannot be spent on core programmes, it may be possible to spend funds in ways that will build capacity: for instance, by hiring additional staff so that the organisation can execute more of the core programme in the future; or by building relationships with new partners.

These three arguments suggest that returns may be diminishing in a continuous manner. This means that returns functions models, or a relative funding gap model, generally seems closer to true returns than the robust-relative funding gap model or the strict funding gap model.

However, models should not only be a close fit to true returns. They should also be clear and easy to communicate, especially for public-facing work. In this respect, an advantage of funding gaps models is that they help facilitate donor coordination. A disadvantage is that they may mislead about the shape of true returns.

Funding gaps models are somewhat easier to explain, especially in a non-technical way. They are also especially useful for facilitating donor coordination: if you say that charity A has a $5m funding gap, and charity B has a $2m funding gap, it is easy for donors to coordinate, by giving in a 5:2 ratio (or by giving to charity A, as long as other donors have given less than $5m, and then giving to charity B). If you instead specify logarithmic returns functions for the two charities, donors may have to get out their calculators to work out where to donate.

So funding gaps models are simple, but they appear more open to misinterpretation than returns functions models. When you are using a relative funding gap model, talk of a funding gap can imply that returns are diminishing rapidly beyond the funding gap, even if this is not what you believe. That is, even if you intend a relative funding gap model, people may think you’re using a strict or robust relative model. This means that although a relative funding gap model may be a close fit to returns, it is liable to mislead. Funding gaps can seem more important than the author meant them to be.

So funding gaps models may facilitate easier understanding and better coordination, but carry a risk of misleading readers about your best model of true returns.

To recap: Returns seem to diminish smoothly in expectation, for three reasons. First, opportunities are likely to be distributed smoothly. Second, we are uncertain about when returns kick in. Third, funds not used in one year can be spent in the next. Funding gaps models fail to model this well: In the case of strict or robust relative models they deny smoothly diminishing returns, whilst relative funding gap models are liable to be misunderstood as denying smoothly diminishing returns. For this reason, returns functions generally seem to be clearer. However funding gaps models may better facilitate coordination, and so may be preferable in certain situations.

As usual, we may face tradeoffs between accurate modelling, simplicity, and clear communication of results, and we may have missed some important considerations in this post. Model selection should be considered carefully on a case-by-case basis, with an eye to the particular features of true returns, and the purpose of your modelling. If you are using the model for communication, you should also consider the audience, and the reason you are writing.^fn-8