Lifetime Impact of a GiveWell Researcher

Entry question

What is could be the lifetime estimated impact (in $ donated) of being a EA GiveWell researcher?

Epistemic Status: Off by a factor 10 (#fermi-estimates)

TL;DR

This is a fermi estimate and shall be treated like that. Everything is an estimate, there are many uncertainties and geometric means have been used to bring the limits down to one number.

Lifetime Impact of a GiveWell Researcher: $8m

Introduction

I am curious to know what sort of impact a EA GiveWell Researcher has. I choose GiveWell, as there is a very “clear” dollar figure on how much money they “influenced” (at least according to them). With this essay, the lifetime impact of an entry level researcher starting in 2021, and working for another 35 years, is estimated. The following components help put together the final figure:

Organization Level

1. Impact of GiveWell in 2021
2. Growth rate of GiveWell’s impact for the next 35 years
3. Counterfactual impact of GiveWell

Researcher Level

1. Impact attributed to an entry level researcher in the first year (2021)
2. Growth rate of researcher’s impact
3. Counterfactual impact of a Researcher

The rest of the essay deals with the above components and the final lifetime impact calculation.

Note1: Fermi estimates are in general factor 2 or even factor 10 off. In many cases this might just be enough.

Note2: This estimate does not consider several factors such as probability of EA loosing its momentum, probability of getting fired and not being able to contribute as a researcher etc.

Impact and Growth Rate of GiveWell in 2021

GiveWell estimates how much money they move every year and publishes them on their website. For example, in the year 2019 GiveWell estimated that they moved $172m. In order to estimate the impact of GiveWell in 2021, a bit of extrapolation of data is needed. For this extrapolation, the growth rate is used.

To predict the growth rate in the coming years I believe it would be reasonable to base it on the money moved from 2015 to 2019 instead of 2007 to 2019. This is because it appears that in the recent years the growth has sort of “plateaued” (see table below) and this might be more representative of the future.

Thus, the compound annual growth rate for the money moved between 2015 and 2019 comes out to be 9.4% ((172/120)^(1/4)-1). It’s really hard to predict the change in growth rate for the coming years. For simplicity’s sake, a constant growth rate of 9.4% for the next 35 years, is assumed.

Impact of GiveWell in 2021

As for the impact of GiveWell in 2021 it is estimated to be $205m (172*(1.094)^2), based on a growth rate of 9.4% and the money moved in 2019.

Counterfactual Impact of GiveWell

The counterfactual impact of GiveWell is the difference in impact between the “Actual Scenario” (world with GiveWell since 2007) and the “Counterfactual Scenario” (world without GiveWell since 2007).

In the counterfactual scenario, it is expected that a GiveWell-like organization would have formed anyway, 3-10 years later (than when GiveWell was founded)². It appears likely because, many “similar” organizations (such as Charity Science Health, Rethink Priorities, Open Philanthropy, The Life You Can Save etc.) have formed since then (between 2010 and 2020), involving cost-effectiveness estimates and philosophies such as “doing the most-good for a dollar”.

While comparing the actual and counterfactual scenario, we compare the two scenarios with the same amount of money but only varying cost-effectiveness depending on the scenario. As a result it is expected that when money doesn’t go to GiveWell-like organizations, it goes to some random organization which is guessed to be within 1 to 1/50th times as cost-effective as GiveWell.

The counterfactual impact for the year 2021 is calculated as follows: As part of the actual scenario, GiveWell is estimated to move $206m with cost-effectiveness of 1. As part of the counterfactual scenario GiveWell-like organization is expected to move $118m ³ with cost-effectiveness of 1, in that year. The remainder $88m (206m - 118m$) is used by “other” organizations at one-seventh (geometric mean of 1 and 1/50) the cost-effectiveness of that of GiveWell. The difference between the actual and the counterfactual situation comes to be $75m. And finally, the counterfactual impact as a function of the money moved in the actual scenario is 36%.

The below table summarizes the counterfactual impact for the different years. For years beyond 2021 the counterfactual impact essentially starts to become a constant value of 31%. This 31% is used in the final calculation as a constant.

AS –> Actual scenario with cost-effectiveness 1
CS GW-like –> Counterfactual scenario with cost-effectiveness 1
CS non-GW-like –> Counterfactual scenario with cost-effectiveness 1/7

Note: Cost of operation of GiveWell is not taken into account as it is a small sum. For example, it is estimated that the cost of operation for 45 people ($110k per person) shall be $5m. Contrast that to it’s 206m expected to be moved in 2021 (a mere 3%). So this is skipped confidently.

Note: Also the effects of the displacement chain are not considered for simplicity’s sake.

Impact Attributed to an Entry Level Researcher in the First Year

In the year 2015, GiveWell estimates that an entry level researcher had 10% of the impact of a co-founder and that there were 9 co-founder equivalents ⁴. In other words an entry level researcher was estimated to be responsible for 1.1% (10%/9) of the total impact (in 2015). But this is not the whole story.

With more people joining GiveWell, this percentage gets diluted. In 2015 there were 25 “equivalent staff members”, where as in 2021 there are 45 “equivalent staff members” ⁵. The resulting contribution would thus be much less. Let’s assume this contribution is inversely proportional to the total number of staff. Then, the impact attributed to an entry level researcher in the first year (2021) is 0.6% (1.1%x25/45).

Growth Rate of a Researcher

The growth rate of a researcher is split into 2 factors. The first being the growth rate associated with the ‘increase in experience’, which leads to higher researcher’s contribution. And the second being the growth rate associated with ‘increase in number of staff’ of GiveWell, which decreases the researcher’s contribution as years pass.

For the “growth rate associated with increase in experience”, the year-variable is held constant (and hence the number of staff is constant) and the difference between entry level researcher and co-founder is observed.

In 2015, a co-founder is estimated to be responsible for 11%⁴ of the impact that year and an entry level researcher at 1.1% (as mentioned above). If it takes an entry level researcher 10-20 years (geometric mean: 14 years) to reach co-founder level, then growth rate per year due to increase in experience (alone) is a whopping 19% ((11%/1.1%)^(1/n)-1). (It is assumed further that the growth rate falls sharply to 0% after reaching co-founder level.)

For the “growth rate attributed to increase in staff”, the average increase in the number of people per year in GiveWell is estimated. This increase is assumed to be negatively proportional to growth rate attributed to increase in staff.

From 2015 to 2021 the number of staff has increased from 25 to 45 ⁵. This implies an increase per year of 10% per year ((45/25)^(1/6)-1). This implies a growth rate attributed to increase in staff of -10%.

Counterfactual Impact of a Researcher

The researcher positions seem very competitive now (with ~1% acceptance rate). If the entry level researcher manages to be 5-15% (geometric mean: 9%) better than the next researcher in line (perhaps by Deliberately Practicing), then that would be the factor multiplied to the impact attributed to a researcher. This would then produce the counterfactual impact of the researcher.

Note: I tried considering a “complex model” taking into account displacement chains (explained in the appendix), but the improvement was almost nothing (few $100k) compared to the final value, so it has been moved to the appendix. Adding it here only made the explanation unnecessarily complicated.

Bringing it all together for 2021 to 2056

The different factors computed so far are listed below:

Organization level

1. Impact of GiveWell in 2021 = $205m

2. Constant growth rate of GiveWell = 9.4%

3. Counterfactual impact = 31%

Researcher Level

1. Impact attributed to an entry level researcher in the first year (2021) = 0.6%

2. Growth Rate due to increase in experience = 19% between 2021 and 2035, and 0 otherwise.

3. Growth Rate due to increase in staff members = -10%

4. Counterfactual contribution of Researcher = 9%

Taking them into account brings the counterfactual impact of a researcher to $8m (could be off by a factor of 10). The following table shows part of the Google Sheets document used to compute the final outcome.

Note: Milan Griffes estimates here that as an entry level researcher he expects to make an impact of $90k (back in 2016). This is one-third the estimate for 2021 as per the above calculations.

Why I can only be a factor 10 confident

First of all these are all forecasts, there are a ton of assumptions, such as the constant growth rates, the entire counterfactual scenario being “made up”, counterfactual impact of a researcher being 9%, etc. I don’t think a more complex model is the solution. Even a really complex model of the growth rate will have it’s own un-testable assumptions by virtue of it being a forecast.

It is very hard to verify this on any level. For example, GiveWell estimates that in 2015 an entry level researcher contributes 10% of a co-founder. I don’t know how accurate this is and how they are testing this assumption over time.

Many numbers are crazy sensitive. Just consider that the counterfactual contribution of a researcher changes to 1% instead of the 9%, then the final estimate falls from $8m to $800k. And we will “never really know” what the “true” value is. The estimation gets even harder with organizations like Rethink Priorities for example, where they produce research and are not really “moving money”.

Appendix: Counterfactual Impact of a Researcher

The counterfactual impact of a researcher is the difference in impact between the “Actual Scenario” where there is an extra researcher and the “Counterfactual Scenario” which is without that extra researcher.

A model table is shown below attempting to estimate the difference in impact between the 2 scenarios. In the actual scenario we have researchers $R_{1}$ to $R_{n}$ and people who Earn-To-Give: $R_{n+1}$ followed by $E_{2}$ to $E_{n}$. In the counterfactual scenario we are missing $R_{1}$ and the whole chain displaces upwards accordingly. EA is the amount of impact associated with being a researcher and ETGX is the Earning-To-Give impact of different people. And the final counterfactual impact is given by $R_{n}-R_{n+1}$x EA + $R_{n+1}$x ETG1.

There are two components to the counterfactual impact of a researcher. The difference between the extra ($R_{1}$) and the displaced researcher is the first one: $R_{n+1}$) ($R_{n}-R_{n+1} x EA$. The researcher positions seem very competitive now (with ~1% acceptance rate). If the extra researcher ($R_{1}$) manages to be 5-15% (geometric mean: 9%) better than the displaced researcher, then the impact will be 0.09 x EA.

In addition, the displaced researcher’s Earning-To-Give ($R_{n+1}$ x ETG1) will also contribute to the counterfactual impact. Assuming the person is European and is willing to donate 10% of his salary, this would roughly mean an Earning-To-Give base of $4000 with a growth rate of 5%.

Footnotes