2f:T589,This notebook is a [Squiggle](https://squiggle-language.com/) model for [GiveWell's GiveDirectly cost-effectiveness analysis](https://docs.google.com/spreadsheets/d/1Kq6iHSQFr3eRz1p9KclHuJTQiaJYkpViOyneSd9KCJc/edit#gid=1680005064). GiveWell represents their GiveDirectly model in a Google Sheet and does not represent uncertainty with their calculations. In contrast, this model handles uncertainties using probability distributions. This notebook is interactive. Changing the parameters and calculations will update the results. It is also a [living document](https://en.wikipedia.org/wiki/Living_document) as you can [leave comments](https://observablehq.com/@observablehq/comments) and [submit changes](https://observablehq.com/@observablehq/fork-suggest-merge). For the most part, this model is a transposition of GiveWell's GiveDirectly model. I add uncertainty to parameters with two methods: - If there was enough information about the parameter, a formal Bayesian update was performed to arrive at a fairly accurate estimate of the uncertainty. - If there wasn't as much information, I guessed the uncertainty with the help of Nuño Sempere, a respected forecaster. These estimates were basic and could be better estimated in the future. I also found [an internal inconsistency](#cell-1830) that changes the output in GiveDirectly's model. You can fix this by checking a box at the bottom of the notebook.30:T4b0,## Motivation for this work GiveWell's CEAs are considered somewhat of a gold standard within EA. This work was originally motivated by wanting to see how far forward we can push that standard. There are a number of things that could be done to improve GiveWell CEAs. This document focuses on Quantified Uncertainty. This notebook attempts to quantify the uncertainty around GiveDirectly's CEA. We do this by letting parameters in the calculations be represented by probability distributions, and seeing how those probability distributions combine to represent uncertainty in the final cost effectiveness. This serves a dual purpose of demonstrating Squiggle. I'm very excited for the possibilities available with Squiggle and Squiggle notebooks. Squiggle notebooks allow you to quantify uncertainty, but also allow you to write and explain the assumptions of the model in text form. They include links, images, tables, and can pull data from other locations. It's also much easier to use and learn than a full probabilistic programming language. I hope that others will be inspired to create their own Squiggle notebooks for their evaluations and estimates, and enjoy the process as much as I did.31:T4d5,## Notes on the implementation To see the code that was used to create this notebook, hover over a cell and click the caret to the left. The text cells will mostly show you markdown, but the Squiggle cells will show you JavaScript. Observable is a JavaScript notebook platorm, and we've used it to embed Squiggle notebooks inside. You can change the text in the cells to see how it would change the model. Please note that currently this isn't that performant. [We are working on improving the performance to make a better experience](https://github.com/quantified-uncertainty/squiggle/issues/465). Changing the text does not change the model for anyone else but yourself, and refreshing the browser will remove your changes. If you wish to persist your changes, you can fork this model by clicking the fork button in the top right of the notebook. Then look into the JavaScript and edit the squiggle string passed into the `cell` or `partial` function. If you wish, you can even suggest changes that I may merge into this one, making this notebook a collaborative document. This is all provided by observable. If you want to learn more about Squiggle notebooks, check out [our tutorial](https://observablehq.com/@hazelfire/squiggle).32:T811,$`\begin{aligned} \text{grant size} &= \frac{\param{household grant size}}{\param{household size}} \\ \text{amount invested} &= \param{portion invested} \cdot \ref{grant size} \\ \text{amount consumed} &= (1 - \param{portion invested}) \cdot \ref{grant size} \\ u(x) &= ${isoelastic_utility ? `\\frac{x^{1 - \\mu} - 1}{1 - \\mu} \\\\ \\mu = 1.59` : `\\log_e x`}\\ \text{initial consumption} & = (\href{#cell-2000}{u}(\frac{\text{\href{#amount_invested}{amount consumed}}}{\param{duration initial consumption}} + \param{baseline consumption}) \\ &- u(\param{baseline consumption})) \cdot \param{duration initial consumption} \\ PV(p, d, t) &= p \int_0^t (1 + d)^{-x} dx = p\frac{1 - (1 + d)^{-t}}{\log_e(1 + d)}\\ \text{value from roi} &= \href{#utility}{u}(\param{return on investment} \cdot \ref{amount invested} + \param{baseline consumption}) \\ &- \href{#utility}{u}(\param{baseline consumption}) \\ \text{value from investment} &= PV(\text{value from roi}, \param{discount rate}, \param{investment duration} - 1) \\ \text{consumption last year} &= \textit{\param{baseline consumption}} + \ref{amount invested} \\ & \cdot (\textit{\param{return on investment}} + \param{portion investment returned})\\ \text{value resell} &= (\href{#utility}{u}(\text{consumption last year}) - u(\param{baseline consumption})) \\ &/ (1 + \param{discount rate})^{\param{investment duration}} \\ \text{total value including spillover} &= \ref{total value} \cdot \param{negative spillover} \\ \text{value per household} &= \text{total value including spillover} \cdot \param{household size} \\ \text{value per dollar} &= \param{transfer efficiency} \cdot \frac{\text{value per household}}{\param{household grant size}} \\ \text{value doubling consumption} &= u(\param{baseline consumption} \cdot 2) - u(\param{baseline consumption}) \\ \text{doubling consumption per dollar} &= \frac{\text{value per dollar}}{\text{value doubling consumption}} \\ \text{value after adjustments} &= \text{doubling consumption per dollar} \cdot \prod_{c \in \param{adjustments}} c \end{aligned} `33:T4bf,The recipient is then modelled to invest the other portion of their grant. The return on investment from the grant was estimated in GiveWell analysis. Here's a discussion from GiveWell: > **Notes from former GiveWell Research Analyst Chris Smith**: I would guess that roofs are an unusually good investment item. I expect that most other assets GiveDirectly's recipients invest in offer lower returns for fewer years. > > It's also important to account for the fact that many items counting as assets (for the sake of the suggested 39% input for "Percentage of transfers invested") aren't interest-bearing (at least in the conventional sense). For example, many people use transfers to buy furniture. > **Notes from GiveWell Senior Fellow Caitlin McGugan**: > Taking the conservative side estimate of the return to iron roofs. I agree with Chris that there’s probably a mix of investments, some of which is in durable consumption goods that don’t earn a return, bringing the average rate of return down. But I also think this is partially compensated by people selecting into non-roof investments (such as livestock) because they believe they’ll be more successful than average in that type of investment. 34:T516,The investment period runs over a duration. This duration is a very uncertain value because follow up studies are yet to be performed by GiveDirectly. Here are some discussions on the value by GiveWell staff: > **Notes on this input from GiveWell Senior Fellow Caitlin McGugan**: > This input is very uncertain as we lack sufficiently long follow-up evidence for GiveDirectly recipients. I'm choosing a value that I think is relatively optimistic but somewhat below the expected lifespan of a roof or a goat. > On the one hand, I think the transfers and some investments made using them can help buffer households from income shocks, prolonging the time until they’re forced to consider selling assets. On the other hand, I think negative income shocks are frequent in these populations, and there’s a good chance that households won’t be able to hang onto their investment over the long term. The balance of evidence I'm aware of from other types of transfer programs suggests that it's challenging for households to maintain asset investments over the very long term. > **Notes on this input from GiveWell Senior Advisor Michael Eddy**: Recent evidence (from Uganda, Ethiopia & Nairobi) makes me generally pessimistic on the long-term (> 5 year) impacts of GiveDirectly-style cash transfers.35:T4c4,The main part of the analysis is done. In the next section, we go through possible positive and negative adjustments to the impact based on other considerations that are not inside the model. Consider all numbers here to be percentages that get multiplied by the final result, like an adjustment of 1 means 100%, and is actually no adjustment. Similar adjustments are considered for all investments, as of such a number of these adjustments are simply not applicable for GiveDirectly. ### Risks of wastage The adjustments are defined by GiveWell as follows: > **double_treatment** The program is delivered to people who don't benefit or minimally benefit because they have received the program from another source. > > Not applicable for GiveDirectly > **ineffective_goods** Pills, LLINs, etc. are poorly made, counterfeit, or otherwise largely or fully ineffective at achieving the expected outcome. > > Not applicable for GiveDirectly > **cash_doesnt_reach_participants** Cash doesn't reach recipients or is taken back from them inappropriately > > we use the amount that GiveDirectly sends to recipients, not adjusted for cash that doesn't ultimately reach recipients through bribery/theft or cases of staff fraud.36:T57a,These considerations are defined by GiveWell as follows: > **change_of_priorities** Charity decides to use some of the funding we directed to them to fund some research or other work that they see as being related to the program, but we don't find valuable. This would affect our cost per output analyses once it happened, so in many cases, no additional adjustment is needed. May be worth adjusting for for charities that we have less experience with. > > Our cost per dollar delivered calculation captures spending on priorities other than delivery of cash transfers. > > **non_funding_bottlenecks** Charity encounters non-funding bottlenecks to allocating funding and holds funding for a long time. At an extreme, charity is unable to spend money, draws down funding over time on work to identify opportunities to spend money for a time, and then shifts to another program. > > Has been able to spend funding quickly. > > **within_org_fungibility** Charity runs multiple programs and shifts funds from non-GiveWell sources or fundraising effort out of the program that we want to fund as a result of receiving GiveWell-directed funding. > > All funds support cash transfer programs, however some programs may be less cost-effective than the program we model in our cost-effectiveness analysis and there may be some fungibility across programs, particularly in how fundraising staff spend their time.37:T67f,viewof read_params = cell(`params(x) = ({ "change_of_priorities": x["Change of priorities"], "non_funding_bottlenecks": x["Non-funding bottlenecks"], "within_org_fungibility": x["Within-org fungibility"], "misappropriation": x["Misappropriation without monitoring results"], "bad_monitoring": x["False monitoring results"], "changes_in_ppp": x["Changes in PPP and exchange rates"], "developmental_effects": x["Developmental effects"], "reduced_morbidity": x["Reduced morbidity"], "child_mortality_effects": x["Child mortality effects"], "double_treatment": x["Double treatment"], "ineffective_goods": x["Ineffective goods"], "cash_doesnt_reach_participants": x["Goods purchased and left in storage until they expire / Cash doesn't reach recipients or is taken back from them inappropriately"], "household_grant_size": x["Total size of transfer per household (nominal USD)"], "household_size": x["Average household size"], "discount_rate": x["Discount rate"], "portion_invested": x["Percentage of transfers invested"], "transfer_efficiency": x["Transfers as a percentage of total cost"], "return_on_investment": x["Return on investment"], "portion_investment_returned": x["Percent of investment returned when benefits end"], "negative_spillover": x["Discount to the benefits of cash from potential negative spillover effects on consumption"], "baseline_consumption": x["Baseline annual consumption per capita (in nominal USD)"], "duration_initial_consumption": 1, "investment_duration": x["Duration of investment benefits (in years)"] }) givewell_params = params($params) 0`, [], {jsImports: {params: param_imports}})38:T74a,viewof my_params = cell(`arbitrary(x) = (0.9 to 1.1) * x uncertain_absolute(x, r) = normal({ p5: x - r, p95: x + r }) dictMap(x, f) = Dict.fromList(map(Dict.keys(x), {|key| [key, f(key, x[key])]})) adjustment_uncertainties = { change_of_priorities: 0.0001, // transfer_efficiency already includes this uncertainty non_funding_bottlenecks: 0.02, // Spends money fairly quickly within_org_fungibility: 0.05, // Possibly funges money through fundraising misappropriation: 0.01, // Very well monitored bad_monitoring: 0.01, changes_in_ppp: 0.05, // Not sure, likely to be small developmental_effects: 0.05, // Developmental effects could make a big change reduced_morbidity: 0.04, // Morbidity could also be a big change child_mortality_effects: 0.04, // Not sure about mortality effects "double_treatment": 0.0001, "ineffective_goods": 0.0001, "cash_doesnt_reach_participants": 0.01 } portion_stdev = { "discount_rate": 0.01, "portion_invested": 0.02, "transfer_efficiency": 0.05, "negative_spillover": 0.03, } lognormal_stdev = { "household_grant_size": 50, "return_on_investment": 0.03, "portion_investment_returned": 0.03, "duration_initial_consumption": 0.3, "investment_duration": 4, "household_size": 0.7, } rest = { "baseline_consumption": SampleSet.fromDist(lognormal({mean: givewell_params.baseline_consumption, stdev: 20}))} adjustment_params = dictMap(adjustment_uncertainties, {|key, v| uncertain_absolute(givewell_params[key], v)}) portion_params = dictMap(portion_stdev, {|key, v| beta({mean: givewell_params[key], stdev: v})}) lognormal_params = dictMap(lognormal_stdev, {|key, v| lognormal({mean: givewell_params[key], stdev: v})}) givedirectly_params = Dict.merge(adjustment_params, Dict.merge(portion_params, Dict.merge(lognormal_params, rest))) 0`, [household_size, baseline_consumption, read_params]) 39:T406,## Discussion Quantifying uncertainty eventually found that the cost effectiveness of GiveDirectly varies by about an order of magnitude and is approximately lognormally distributed. The cost effectiveness however doesn't have particularly large tails. The benefits of the quantification of uncertainty have not yet been fully realised with this document. From this, it could be possible to calculate the [value of information](https://en.wikipedia.org/wiki/Value_of_information). In the future, I would love to continue this work in quantifying uncertainty for the rest of the GiveWell models, or possibly bringing back quantification for the [ACE models](https://animalcharityevaluators.org/charity-reviews/evaluating-charities/evaluation-criteria/#ce). I had a lot of fun writing this Squiggle Notebook. It offers a large collection of advantages over other alternatives: - Ability to write detail inbetween cells - External links - Ability to quantify uncertainty - Ability to create calculators and interactive analysis 3a:T5be,### Isoelastic Utility The [calculated discount rate assumes](https://docs.google.com/document/d/1g2tf7Fu8-iG_8TgnJkEx9G5TlEyrcv7PXPokmPZaUMc/edit) that welfare is an [isoelastic](https://en.wikipedia.org/wiki/Isoelastic_utility) to consumption with ${tex`\eta=1.59`}, but the rest of the model assumes that welfare is isoelastic to consumption with ${tex`\eta = 1`} (aka, logarithmic). [To quote](https://docs.google.com/document/d/1g2tf7Fu8-iG_8TgnJkEx9G5TlEyrcv7PXPokmPZaUMc/edit): > Increases in consumption over time meaning marginal increases in consumption in the future are less valuable. We chose a rate of 1.7% based on an expectation that economic consumption would grow at 3% each year, and the function through which consumption translates to welfare is [isoelastic](https://en.wikipedia.org/wiki/Isoelastic_utility) with eta=1.59. Considering ${tex`\eta`} has been [estimated as 1.5 in the United Kindom](https://link.springer.com/article/10.1007/s10640-018-0242-z) and [1.4 in other OECD countries](https://www.jstor.org/stable/24440019), this model should also probably consider welfare to be an isoelastic function of consumption. This would increase the effectiveness of cash transfers. I've created a [desmos calculator to explore this concept](https://www.desmos.com/calculator/a0xtpziga4). Changing the utility function to be isoelastic is an enourmous change. **Fixing this increases GiveDirectly's cost effectiveness by a factor of 11%**.3b:T4d5,1. This model sometimes double counts consumption increases. For instance, if the duration of the and the investment returns come during the same period of time, however the increases of consumption during that period come from the same baseline, therefore double counting the increase in marginal consumption over the first year. I don't believe fixing this problem would change the results that much, and decided not to include it. 2. The adjustments section do not have any uncertainty associated with them, even though these parts are probably the most uncertain. These adjustments are also moraly uncertain. 3. There are two other people who have done extremely similar work, [Cole Haus](https://forum.effectivealtruism.org/posts/gMxTEMvh8RttX9Nt4/uncertainty-and-sensitivity-analyses-of-givewell-s-cost) and [Joel McGuire](https://www.happierlivesinstitute.org/report/cash-transfers-cost-effectiveness-analysis/). It would be valuable to compare this work to their results. 4. It would be good to find out exactly how much uncertainty comes from our estimates vs more formal bayesian analysis. 5. It could be possible to look into the value of further research in GiveDirectly based on comparing it to other GiveWell interventions.3c:T451,appendixc = md`## Appendix B, Notes on Bayesian Inference Most parameter choices here are based on estimates. I worked with Nuño Sempere to refine these estimates and make sure they were appropriate. However, there were two parameters (household sizes and baseline consumption) which we had enough data to do Bayesian inference on. This appendix goes over the methods used to run the Bayesian inference. Household sizes were assumed to follow a [Poisson Distribution](https://en.wikipedia.org/wiki/Poisson_distribution). A Poisson distribution takes one parameter, ${tex`\lambda`}, that is the expected value of the distribution. Because we care about the distribution of the expected value, we want to find out what the distribution of ${tex`\lambda`} is based on the available data. For baseline consumption, I assumed that income was lognormally distributed. However, if you log a lognormal distribution you arrive at a normal distribution. So I simply log the observations and run bayesian inference over a normal distribution. Details on the math are provided on the following external notebook.`3d:T425,compare_givewell = (name, cell, index) => { let formatPercent = (x) => new Intl.NumberFormat('en-US', {style: 'percent', maximumSignificantDigits: 3, signDisplay: "always"}).format(x) let result = project.getResult(cell).value let env = project.getEnvironment() let givewell = parseFloat(gw_gd_cea.filter(x => x["Cash Transfers - GiveDirectly"] === name)[index ?? 0]["Overall"].replace(",","")) if(result.tag === "Number"){ let value = result.value; let difference = (value - givewell) / givewell; return Inputs.table([{ "GiveWell": formatFloat(givewell) ,"Value": formatFloat(value), "Difference": formatPercent(difference)}]) } else { let dist = result.value let mean = dist.mean(env).value let lowCI = dist.inv(env, 0.025).value let highCI = dist.inv(env, 0.975).value let difference = (mean - givewell) / givewell return Inputs.table([{ "GiveWell": formatFloat(givewell) ,"Mean": formatFloat(mean), "95% CI": `${formatFloat(lowCI)} - ${formatFloat(highCI)}`, "Difference": formatPercent(difference)}]) } }3e:[{"id":"8ad0773214d8e7f14ef0a9adcc4679f7e6e1bd9a5f9a6037dbb852d52a33c0f68537840d1c83df3fd2ed396a66edb1bd472f694308052c3e50c3c04ed354c198","href":"https://static.observableusercontent.com/files/8ad0773214d8e7f14ef0a9adcc4679f7e6e1bd9a5f9a6037dbb852d52a33c0f68537840d1c83df3fd2ed396a66edb1bd472f694308052c3e50c3c04ed354c198","name":"household size analysis - Family size by household.csv","type":"text/csv","lastModified":1667093130408.159,"size":1634},{"id":"cabadb0a43d285c8374d6031d96779e0d7bb43f4841026b58605bf1fbf06279fa6aed08a46a6b657a4c9b6ab637bece0251be36658cb3e5fa3321be04ef3d579","href":"https://static.observableusercontent.com/files/cabadb0a43d285c8374d6031d96779e0d7bb43f4841026b58605bf1fbf06279fa6aed08a46a6b657a4c9b6ab637bece0251be36658cb3e5fa3321be04ef3d579","name":"gw_gd_cea.csv","type":"text/csv","lastModified":1667093130408.159,"size":4124}] 2d:{"title":"Quantifying Uncertainty in GiveWell's GiveDirectly Cost Effectiveness Analysis","cells":[{"id":124,"value":"# Quantifying Uncertainty in GiveWell's GiveDirectly Cost Effectiveness Analysis","pinned":false,"hidden":false,"mode":"md"},{"id":1557,"value":"**Disclaimer**: I (Sam Nolan) do not represent GiveWell or the Quantfied Uncertainty Research Institute. All opinions and mistakes are my own. This notebook is referenced as as a supplemental notebook for a discussion on [Quantifying Uncertainty in GiveWell CEAs](https://observablehq.com/@hazelfire/quantifying-uncertainty-in-givewell-ceas) which is being submitted for consideration in [GiveWell's *Change our Mind Contest*](https://www.givewell.org/research/change-our-mind-contest#:~:text=GiveWell%20is%20launching%20a%20contest,lives%20the%20most%20per%20dollar.)","pinned":false,"hidden":false,"mode":"md"},{"id":1518,"value":"toc()","pinned":false,"hidden":false,"mode":"ojs"},{"id":407,"value":"## Introduction","pinned":false,"hidden":false,"mode":"md"},{"id":58,"value":"$2f","pinned":false,"hidden":false,"mode":"md"},{"id":641,"value":"## What is Squiggle?\nThis notebook uses Squiggle to do calculations using uncertainty. \n\n[Squiggle](https://squiggle-language.com/) is a simple estimation programming language for forecasters and evaluators. It's in a pre-alpha stage, and comes with a healthy [collection of bugs](https://github.com/quantified-uncertainty/squiggle/issues?q=is%3Aissue+is%3Aopen+label%3Abug+) and [missing features](https://github.com/quantified-uncertainty/squiggle/discussions). Squiggle is made by the [Quantified Uncertainty Research Institute](https://quantifieduncertainty.org/).\n\nIf you want to use Squiggle, I recommend checking out the [playground](https://www.squiggle-language.com/playground/). For Observable support, check out the [demo Squiggle notebook](https://observablehq.com/@hazelfire/squiggle). Also feel free to directly get in contact with me via my [calendly](https://calendly.com/sam_nolan/30min) or the [EA Forum](https://forum.effectivealtruism.org/users/hazelfire).","pinned":false,"hidden":false,"mode":"md"},{"id":646,"value":"$30","pinned":false,"hidden":false,"mode":"md"},{"id":674,"value":"$31","pinned":false,"hidden":false,"mode":"md"},{"id":324,"value":"import {cellGen, newProject, stylesheet, saveProject, runGen, exportSampleSet} from '@hazelfire/squiggle'","pinned":false,"hidden":false,"mode":"ojs"},{"id":2923,"value":"saveProject(project)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1898,"value":"stylesheet","pinned":false,"hidden":false,"mode":"ojs"},{"id":2831,"value":"project = newProject()","pinned":false,"hidden":false,"mode":"ojs"},{"id":2835,"value":"cell = cellGen(project)","pinned":false,"hidden":false,"mode":"ojs"},{"id":3016,"value":"run = runGen(project)","pinned":false,"hidden":false,"mode":"ojs"},{"id":651,"value":"# GiveDirectly model","pinned":false,"hidden":false,"mode":"md"},{"id":1567,"value":"GiveDirectly creates wellbeing through direct cash transfers. In [GiveWell's model](https://www.givewell.org/how-we-work/our-criteria/cost-effectiveness/cost-effectiveness-models), wellbeing created is measured by doublings of consumption. According to the model, if you double the consumption of an individual for a year, you get 1 unit of value.\n\nThe model starts by working out how much of your donation goes to each individual. First by subtracting charity costs, then dividing the rest among households, then dividing that among every member of the household.\n\nFrom there, the recipient is modelled to invest a portion of their money and use a portion of their money. These effects are then added together. This creates the amount of value before adjustments.\n\nFinally, the value is adjusted for other adjustments that are not included within the model. For instance, the likelihood on whether this could decrease child mortality.","pinned":false,"hidden":false,"mode":"md"},{"id":2650,"value":"For the brave hearted, the full formula of the model is below. Italics represent parameters, and each term will link to where it is defined in Squiggle.","pinned":false,"hidden":false,"mode":"md"},{"id":2591,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":413,"value":"## Transfer size","pinned":false,"hidden":false,"mode":"md"},{"id":1964,"value":"$$`\\text{grant size} = \\frac{\\param{household grant size}}{\\param{household size}}`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1969,"value":"viewof grant_size = cell(`grant_size(x) = x.household_grant_size / x.household_size\ngrant_size(givedirectly_params)`, [givedirectly_params], {tickFormat: \"$,f\", title: \"Size of transfer per person (USD)\"})","pinned":false,"hidden":false,"mode":"ojs"},{"id":3144,"value":"compare_givewell(\"Size of transfer per person (nominal USD)\", grant_size)","pinned":false,"hidden":false,"mode":"ojs"},{"id":188,"value":"The first important measure is how much money goes to a household. A GiveDirectly grant is usually [given to a household](https://www.givewell.org/charities/give-directly#Overview). GiveDirectly aims to give different amounts to different demographics based on inflation and other considerations. GiveWell gives a point estimate of $1000 based on talking with GiveDirectly:\n\n> [Transfer sizes for the standard lump sum projects: Kenya: $1,085, Rwanda: $970, Uganda: $963.](https://www.givewell.org/charities/give-directly/supplementary-information#footnote2_0h0watm)\n>\n> February 20, 2018, [Joe Huston](https://www.givedirectly.org/author/joe-huston/) GiveDirectly CFO","pinned":false,"hidden":false,"mode":"md"},{"id":194,"value":"viewof household_grant_size = cell(`givedirectly_params.household_grant_size`, givedirectly_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":349,"value":"The money given to a participant is shared among a household. We measure how much good we do for each person, so we'll need the average household size. A point estimate of 4.7 is provided by GiveWell from their [analysis on household size](https://files.givewell.org/files/DWDA%202009/GiveDirectly/household%20size%20analysis.xls) which uses data from the data from the Siaya 2012 census. I made a copy of this spreadsheet (You can see and download the copy by clicking on the paperclip on the right) and used it to do infer the mean household size. More details about this can be seen in [Appendix C](#appendixc).","pinned":false,"hidden":false,"mode":"md"},{"id":420,"value":"## Recipient usage of funds","pinned":false,"hidden":false,"mode":"md"},{"id":198,"value":"Recipients can choose to spend the money now or invest it in something that gives them returns in the future. The point estimate for GiveWell's estimate is that participants invest 39% of the grant. An investment in this case is simply a purchase that has a payoff over a period of time, with a common example being tin roofs or livestock. This figure is retrieved from [Haushofer & Shapiro 2013](https://files.givewell.org/files/DWDA%202009/Interventions/Cash%20Transfers/haushofer_shapiro_uct_2013.11.16.pdf). Pg 20.\n> \"Households invest part of the transfers: the value of non-land assets increased by USD 279 on average; this represents 58 percent of the control group mean USD 478, and 39 percent of the average transfer.\"","pinned":false,"hidden":false,"mode":"md"},{"id":423,"value":"### Amount invested / consumed\n\n${$`\\begin{aligned}\n\\text{amount invested} &= \\param{portion invested} \\cdot \\ref{grant size} \\\\\n\\text{amount consumed} &= (1 - \\param{portion invested}) \\cdot \\ref{grant size}\n\\end{aligned}\n`}\n","pinned":false,"hidden":false,"mode":"md"},{"id":1989,"value":"viewof amount_invested = cell(`amount_invested(x) = x.portion_invested * grant_size(x)\namount_consumed(x) = (1 - x.portion_invested) * grant_size(x)\n{invested: amount_invested(givedirectly_params), consumed: amount_consumed(givedirectly_params)}`, [grant_size, givedirectly_params])","pinned":false,"hidden":false,"mode":"ojs"},{"id":206,"value":"viewof portion_invested = cell(`givedirectly_params.portion_invested`, givedirectly_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":2807,"value":"### Utility\nThe default utility function is logarithmic, but can be changed to isoelastic below. This means that people who are richer value marginal increases in money less than people that are poorer, a core assumption when giving to the poor.","pinned":false,"hidden":false,"mode":"md"},{"id":2000,"value":"$${tex`u(x) = ${isoelastic_utility ? `\\\\frac{x^{1 - \\\\mu} - 1}{1 - \\\\mu} \\\\\\\\ \\\\mu = 1.59` : `\\\\log_e x`}`}","pinned":false,"hidden":false,"mode":"md"},{"id":2002,"value":"viewof utility = cell(`utility(x) = ${isoelastic_utility ? `(x ^ (1 - 1.59) - 1) / (1 - 1.59)` : `log(x)`}`)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1376,"value":"### Log increase in initial consumption","pinned":false,"hidden":false,"mode":"md"},{"id":1952,"value":"$${$`\\begin{aligned}\n\\text{initial consumption} & = (\\href{#cell-2000}{u}(\\frac{\\text{\\href{#amount_invested}{amount consumed}}}{\\param{duration initial consumption}} + \\param{baseline consumption}) \\\\ \n&- u(\\param{baseline consumption})) \\cdot \\param{duration initial consumption}\n\\end{aligned}`}","pinned":false,"hidden":false,"mode":"md"},{"id":2231,"value":"viewof value_initial_consumption = cell(`value_initial_consumption(x) = (utility(amount_consumed(x)/x.duration_initial_consumption + x.baseline_consumption) - utility(x.baseline_consumption)) * x.duration_initial_consumption\nvalue_initial_consumption(givedirectly_params)`, [utility, amount_invested, givedirectly_params]) ","pinned":false,"hidden":false,"mode":"ojs"},{"id":5302,"value":"compare_givewell(\"Total immediate increase in ln(consumption) per person made possible by funds transferred\", value_initial_consumption)","pinned":false,"hidden":false,"mode":"ojs"},{"id":230,"value":"One way that the recipient gets value is by spending the money that they receive immediately. The initial consumption period is implicitly one year in GiveWell's analysis. This parameter is implicit within GiveWell's analysis. Here it's made explicit. From this, we can calculate how much utility we create by subtracting the utility gained from the grant from the utility of the baseline consumption.","pinned":false,"hidden":false,"mode":"md"},{"id":239,"value":"viewof duration_initial_consumption = cell(`givedirectly_params.duration_initial_consumption`, givedirectly_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":430,"value":"### Recipient Investments\n${$`\\begin{aligned}\npv(p, d, t) &= p \\int_0^t (1 + d)^{-x} dx = p\\frac{1 - (1 + d)^{-t}}{\\log_e(1 + d)}\\\\\n\\text{value from roi} &= \\href{#utility}{u}(\\param{return on investment} \\cdot \\ref{amount invested} + \\param{baseline consumption}) \\\\ &- \\href{#utility}{u}(\\param{baseline consumption}) \\\\\n\\text{value from investment} &= pv(\\text{value from roi}, \\param{discount rate}, \\param{investment duration} - 1)\n\\end{aligned}`}","pinned":false,"hidden":false,"mode":"md"},{"id":2252,"value":"viewof value_investment = cell(`pv(payment, discount, duration) = payment * (1 - (1 + discount) ^ (-duration)) / ${continuous_present_value ? `log(1 + discount)` : `discount`}\nvalue_from_roi(x) = utility(x.return_on_investment * amount_invested(x) + x.baseline_consumption) - utility(x.baseline_consumption)\nvalue_investment(x) = pv(value_from_roi(x), x.discount_rate, x.investment_duration - 1)\nvalue_investment(givedirectly_params)`, [utility, amount_invested, givedirectly_params])","pinned":false,"hidden":false,"mode":"ojs"},{"id":3100,"value":"compare_givewell(\"Present value of future increases in ln(consumption) per person (excluding the last year of expected benefits)\", value_investment)","pinned":false,"hidden":false,"mode":"ojs"},{"id":438,"value":"$33","pinned":false,"hidden":false,"mode":"md"},{"id":258,"value":"viewof return_on_investment = cell(`givedirectly_params.return_on_investment`, givedirectly_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":454,"value":"$34","pinned":false,"hidden":false,"mode":"md"},{"id":281,"value":"viewof investment_duration = cell(`givedirectly_params.investment_duration`, givedirectly_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":479,"value":"GiveWell has a [full discussion of their discount rate elsewhere](https://docs.google.com/document/d/1g2tf7Fu8-iG_8TgnJkEx9G5TlEyrcv7PXPokmPZaUMc/edit?usp=sharing). Note that the discount rate is not because there is an inherent value on getting utility now, but is made from three components:\n- Uncertainty in the future\n- Increase in monetary consumption due to other effects, making marginal log increases in consumption later more difficult.\n- Increase in non-monetary consumption, again making marginal log increases in consumption more difficult.","pinned":false,"hidden":false,"mode":"md"},{"id":467,"value":"viewof discount_rate = cell(`givedirectly_params.discount_rate`, givedirectly_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":485,"value":"This is a continuous present value formula, it is different from the one used by GiveWell. I prefer it because I think GiveDirectly investments are probably not well modelled by discrete payouts^{[6](#footnotecontinuouspv)}. It comes from the following formula for present value:","pinned":false,"hidden":false,"mode":"md"},{"id":2407,"value":"cell(`value_investment(givedirectly_params)`, [givedirectly_params, value_investment])","pinned":false,"hidden":false,"mode":"ojs"},{"id":561,"value":"Where is ${tex`p`} the payment, ${tex`t`} is the duration, ${tex`d`} is the discount_rate ","pinned":false,"hidden":false,"mode":"md"},{"id":503,"value":"### Benefits from reselling investments\n${$`\n\\begin{aligned}\n\\text{consumption last year} &= \\textit{\\param{baseline consumption}} + \\ref{amount invested} \\\\\n & \\cdot (\\textit{\\param{return on investment}} + \\param{portion investment returned})\\\\\n\\text{value resell} &= (\\href{#utility}{u}(\\text{consumption last year})\n - u(\\param{baseline consumption})) \\\\\n&/ (1 + \\param{discount rate})^{\\param{investment duration}}\n\\end{aligned}\n`}\n","pinned":false,"hidden":false,"mode":"md"},{"id":2277,"value":"viewof value_resell = cell(`value_resell(x) = (utility(x.baseline_consumption + amount_invested(x) * (x.return_on_investment + x.portion_investment_returned)) - utility(x.baseline_consumption)) / ((1 + x.discount_rate) ^ x.investment_duration)\nvalue_resell(givedirectly_params)`,[utility, amount_invested, givedirectly_params])","pinned":false,"hidden":false,"mode":"ojs"},{"id":5308,"value":"compare_givewell(\"Present value of ln(consumption) increase in the last year of expected benefits\", value_resell)","pinned":false,"hidden":false,"mode":"ojs"},{"id":2275,"value":"GiveWell then considers the benefits from reselling investments. Comments from GiveWell:\n> In some cases, items purchased with cash transfers might be resold when they cease providing returns. We are extremely uncertain about the appropriate value for this parameter. It's our best guess that most investment items will not retain much of their original value. We have not prioritized a thorough investigation into possible values for this parameter.^{[7](#footnotesellinginvestments)}","pinned":false,"hidden":false,"mode":"md"},{"id":290,"value":"viewof percent_investment_returned = cell(`givedirectly_params.portion_investment_returned`, givedirectly_params);","pinned":false,"hidden":false,"mode":"ojs"},{"id":508,"value":"We can then add the three different sources of value together to create the total increase in pv from cash transfers.","pinned":false,"hidden":false,"mode":"md"},{"id":298,"value":"viewof total_value = cell(`total_value(x) = value_initial_consumption(x) + value_investment(x) + value_resell(x)\ntotal_value(givedirectly_params)`, [value_initial_consumption, value_investment, value_resell, givedirectly_params])","pinned":false,"hidden":false,"mode":"ojs"},{"id":3058,"value":"compare_givewell(\"Total present value of cash transfer in terms of increases in ln(consumption)\", total_value)","pinned":false,"hidden":false,"mode":"ojs"},{"id":2637,"value":"$$`\\text{total value} = \\ref{initial consumption} + \\ref{returns from investment} + \\ref{value resell}`","pinned":false,"hidden":false,"mode":"ojs"},{"id":545,"value":"## Negative Spillover and Total Value before adjustments\n${$`\\begin{aligned}\n\\text{total value including spillover} &= \\ref{total value} \\cdot \\param{negative spillover} \\\\\n\\text{value per household} &= \\text{total value including spillover} \\cdot \\param{household size} \\\\\n\\text{value per dollar} &= \\param{transfer efficiency} \\cdot \\frac{\\text{value per household}}{\\param{household grant size}} \\\\\n\\text{value doubling consumption} &= u(\\param{baseline consumption} \\cdot 2) - u(\\param{baseline consumption}) \\\\\n\\text{doubling consumption per dollar} &= \\frac{\\text{value per dollar}}{\\text{value doubling consumption}}\n\\end{aligned}\n`}","pinned":false,"hidden":false,"mode":"md"},{"id":5319,"value":"viewof donation_size = cell(`donation_size = 100000`)","pinned":true,"hidden":false,"mode":"ojs"},{"id":2298,"value":"viewof doubling_consumptions = cell(`total_value_including_spillover(x) = total_value(x) * (1 - x.negative_spillover)\nvalue_per_household(x) = total_value_including_spillover(x) * x.household_size\ndonation_size = 100000\ntransfers_made(x) = donation_size / x.household_grant_size\nln_consumptions_per_dollar(x) = x.transfer_efficiency * value_per_household(x) * transfers_made(x)\nvalue_doubling_consumption(x) = utility(x.baseline_consumption * 2) - utility(x.baseline_consumption)\ndoubling_consumptions(x) = ln_consumptions_per_dollar(x) / value_doubling_consumption(x)\ndoubling_consumptions(givedirectly_params)`, [total_value, utility, givedirectly_params, donation_size])","pinned":false,"hidden":false,"mode":"ojs"},{"id":3053,"value":"compare_givewell(\"Total units of value generated\", doubling_consumptions)","pinned":false,"hidden":false,"mode":"ojs"},{"id":529,"value":"The negative spillover represents possible negative externalities that come from direct cash transfers. GiveWell has an [analysis of negative spillover from cash transfers](https://docs.google.com/document/d/1C4nX3LWM-TeNMFxmNAKRbi4vtt_4ZaLoIGZzlaHmvuI/edit#).","pinned":false,"hidden":false,"mode":"md"},{"id":512,"value":"viewof negative_spillover = cell(`givedirectly_params.negative_spillover`, givedirectly_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":177,"value":"The proportion of funding available is the funding that ends up in the hands of grantees, after any required administration costs. This value is given a point value of 83% by GiveWell, and is calculated from [an analysis of GiveDirectly spending in February 2018](https://docs.google.com/spreadsheets/d/1L03SQuAeRRfjyuxy20QIJByOx6PEzyJ-x4edz2fSiQ4/edit#gid=537899494). Based on this analysis, with the help of Nuño Sempere, I constructed an estimate of a beta distribution wih ${tex`\\mu=0.83`} and ${tex`\\sigma = 0.05`} ^{[1](#footnoteto)} for GiveDirectly's efficiency.^{[2](#footnotetransfer)}","pinned":false,"hidden":false,"mode":"md"},{"id":179,"value":"viewof transfer_efficiency = cell(`givedirectly_params.transfer_efficiency`, givedirectly_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":539,"value":"Finally, we total up the amount of value created for each household/transfer","pinned":false,"hidden":false,"mode":"md"},{"id":849,"value":"## Charity level adjustments\n${$`\\text{value after adjustments} = \\prod^{c \\in \\param{adjustments}} c \\cdot \\text{doubling consumption per dollar}`}\n\n","pinned":false,"hidden":false,"mode":"md"},{"id":2323,"value":"viewof value_after_adjustments = cell(`charity_adjustments(x) = 1 - (x.double_treatment + x.ineffective_goods + x.cash_doesnt_reach_participants + x.misappropriation + x.bad_monitoring + x.change_of_priorities + x.non_funding_bottlenecks + x.within_org_fungibility)\nintervention_adjustments(x) = 1 + x.changes_in_ppp + x.developmental_effects + x.reduced_morbidity + x.child_mortality_effects\nvalue_after_adjustments(x) = doubling_consumptions(x) * intervention_adjustments(x) * charity_adjustments(x)\nvalue_after_adjustments(givedirectly_params)`, [doubling_consumptions, givedirectly_params])","pinned":false,"hidden":false,"mode":"ojs"},{"id":3173,"value":"compare_givewell(\"Total units of value generated, after accounting for all adjustments\", value_after_adjustments)","pinned":false,"hidden":false,"mode":"ojs"},{"id":2321,"value":"$35","pinned":false,"hidden":false,"mode":"md"},{"id":852,"value":"viewof risk_of_wastage_adjustments = cell(`{ double_treatment: givedirectly_params.double_treatment, ineffective_goods: givedirectly_params.ineffective_goods, cash_doesnt_reach_participants: givedirectly_params.cash_doesnt_reach_participants}`, givedirectly_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":859,"value":"### Quality of monitoring and evaluation","pinned":false,"hidden":false,"mode":"md"},{"id":1062,"value":"md`These adjustments are described by GiveWell as follows:\n> **misappropriation** Charity sends money/goods to a country/partner and the country/partner reports back that they carried out the program and reached a certain number of people, but these results are partially or fully fabricated and we do not have monitoring results from those locations.\n> \n> We've consistently received comprehensive monitoring results from GiveDirectly. (We chose not to revisit the GiveDirectly's monitoring in 2019 and therefore have not updated this.)\n\n> **false_monitoring** The monitoring results that we've received are partially or fully fabricated or significantly biased in some way we are unaware of, leading us to believe that results are better than they actually were.\n> \n> We have relatively high confidence in GiveDirectly's monitoring, which involves multiple rounds of checks and audits by different teams. (We chose not to revisit the GiveDirectly's monitoring in 2019 and therefore have not updated this.)`","pinned":false,"hidden":false,"mode":"ojs"},{"id":862,"value":"viewof monitoring_and_evaluation_adjustments = cell(`{ misappropriation: givedirectly_params.misappropriation, bad_monitoring: givedirectly_params.bad_monitoring}`, givedirectly_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":868,"value":"### Confidence of funds being used for intended purpose","pinned":false,"hidden":false,"mode":"md"},{"id":1072,"value":"$36","pinned":false,"hidden":false,"mode":"md"},{"id":870,"value":"viewof confidence_of_funds_adjustments = cell(`arbitrary(x) = (0.9 to 1.1) * x\n{ change_of_priorities: givedirectly_params.change_of_priorities, non_funding_bottlenecks: givedirectly_params.non_funding_bottlenecks, within_org_fungibility: givedirectly_params.within_org_fungibility}`, givedirectly_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":909,"value":"## Intervention Level Adjustments","pinned":false,"hidden":false,"mode":"md"},{"id":1076,"value":"GiveWell then specifies [possible effects that are not included in the model](https://docs.google.com/spreadsheets/d/1Kq6iHSQFr3eRz1p9KclHuJTQiaJYkpViOyneSd9KCJc/edit#gid=819415156). Due to lack of strong evidence, none of these effects are considered likely to be large.","pinned":false,"hidden":false,"mode":"md"},{"id":913,"value":"viewof adjustments = cell(`{ changes_in_ppp: givedirectly_params.changes_in_ppp, developmental_effects: givedirectly_params.developmental_effects, reduced_morbidity: givedirectly_params.reduced_morbidity, child_mortality_effects: givedirectly_params.child_mortality_effects}`, givedirectly_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":3104,"value":"$37","pinned":false,"hidden":false,"mode":"ojs"},{"id":5326,"value":"viewof givewell_params = cell(`givedirectly_params = givewell_params\n0`, [read_params])","pinned":false,"hidden":false,"mode":"ojs"},{"id":5334,"value":"viewof arbitrary_params = cell(`arbitrary(x) = x == 0 ? 0 : (x > 0 ? lognormal({mean: x, stdev: x / 3}): -1 * lognormal({mean: -1 * x, stdev: -1 * x / 3}))\ndictMap(x, f) = Dict.fromList(map(Dict.keys(x), {|key| [key, f(x[key])]}))\ngivedirectly_params = dictMap(givewell_params, arbitrary)\n0`, read_params)","pinned":false,"hidden":false,"mode":"ojs"},{"id":2336,"value":"$38","pinned":false,"hidden":false,"mode":"ojs"},{"id":2365,"value":"viewof final_value_per_dollar = cell(`final_value_per_dollar(x) = value_after_adjustments(x) / donation_size\nfinal_value_per_dollar(givedirectly_params)`, [givedirectly_params, value_after_adjustments, donation_size])","pinned":false,"hidden":false,"mode":"ojs"},{"id":3043,"value":"compare_givewell(\"Units of value generated per philanthropic dollar spent, after accounting for all adjustments\", final_value_per_dollar)","pinned":false,"hidden":false,"mode":"ojs"},{"id":3027,"value":"exportSampleSet(project, `final_value_per_dollar(givedirectly_params)`, [final_value_per_dollar,givedirectly_params])","pinned":false,"hidden":false,"mode":"ojs"},{"id":2973,"value":"sensitivity(`value_after_adjustments`, `givedirectly_params`, [givedirectly_params, value_after_adjustments])","pinned":false,"hidden":false,"mode":"ojs"},{"id":1746,"value":"Equivalently, the mean cost per doubling of consumption is as below:","pinned":false,"hidden":false,"mode":"md"},{"id":2392,"value":"viewof dollars_per_double = cell(`1 / value_after_adjustments(givedirectly_params)`, [givedirectly_params, value_after_adjustments], {showSummary: true, tickFormat: \"$,f\", title: \"Cost of doubling someone's consumption\"})","pinned":false,"hidden":false,"mode":"ojs"},{"id":777,"value":"$39","pinned":false,"hidden":false,"mode":"md"},{"id":1370,"value":"### Inference of baseline consumption","pinned":false,"hidden":false,"mode":"md"},{"id":220,"value":"We measure the increase in welfare by doubling consumption. The poorer a recipient the more effective cash transfers become. The baseline consumption for recipients is given as 285.92 USD per year by GiveWell. To quote GiveWell:\n\n> Taken from [Haushofer and Shapiro 2013](https://files.givewell.org/files/DWDA%202009/Interventions/Cash%20Transfers/haushofer_shapiro_uct_2013.11.16.pdf), Pg 49. Non-durable expenditure (USD) in the control group is measured as 157.40 a month. This is multiplied by 12 to get an annual estimated, adjusted for purchasing power parity, and divided by average household size. See [calculations](https://files.givewell.org/files/DWDA%202009/Interventions/GiveWell_cost-effectiveness_analysis_2016.xlsx) [Parameters > Cash] for more details.\n\nI calculate the expected value of baseline consumption through Bayesian Inference, see [Appendix C](#appendixc) for more details.","pinned":false,"hidden":false,"mode":"md"},{"id":1282,"value":"First, we get the mean consumption from the [Haushofer and Shapiro 2013](https://files.givewell.org/files/DWDA%202009/Interventions/Cash%20Transfers/haushofer_shapiro_uct_2013.11.16.pdf), Pg 49","pinned":false,"hidden":false,"mode":"md"},{"id":1141,"value":"monthly_consumption_per_household_ppp = 157.40","pinned":false,"hidden":false,"mode":"ojs"},{"id":1290,"value":"To convert it out of [PPP](https://en.wikipedia.org/wiki/Purchasing_power_parity), we use a multiplier. This value was taken from [GiveWell's parameter calculations](https://files.givewell.org/files/DWDA%202009/Interventions/GiveWell_cost-effectiveness_analysis_2016.xlsx). To quote:\n> Derived using the fact that the authors equate 1085 USD with 1,525 PPP","pinned":false,"hidden":false,"mode":"md"},{"id":1143,"value":"ppp_multiplier = 1.405529954","pinned":false,"hidden":false,"mode":"ojs"},{"id":1297,"value":"We then get the mean household size.^{[3](#footnotemean)}","pinned":false,"hidden":false,"mode":"md"},{"id":2856,"value":"viewof mean_household_size = cell(`mean(household_size)`, household_size)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1148,"value":"household_size_mean = 4.7","pinned":true,"hidden":false,"mode":"ojs"},{"id":1304,"value":"We can then calculate the consumption per person per year that GiveWell arrived at.","pinned":false,"hidden":false,"mode":"md"},{"id":1154,"value":"consumption_per_person_per_year_usd = monthly_consumption_per_household_ppp / household_size_mean / ppp_multiplier * 12","pinned":true,"hidden":false,"mode":"ojs"},{"id":1312,"value":"To do bayesian inference, we'll need the number of samples that was done in the study. Retrieved from [Haushofer and Shapiro 2013](https://files.givewell.org/files/DWDA%202009/Interventions/Cash%20Transfers/haushofer_shapiro_uct_2013.11.16.pdf), Pg 49.","pinned":false,"hidden":false,"mode":"md"},{"id":1140,"value":"consumption_sample_count = 1372","pinned":false,"hidden":false,"mode":"ojs"},{"id":1318,"value":"We also need the standard deviation of the sample, also based on [Haushofer and Shapiro 2013](https://files.givewell.org/files/DWDA%202009/Interventions/Cash%20Transfers/haushofer_shapiro_uct_2013.11.16.pdf), Pg 49.^{[4](#footnoteknownvariance)}","pinned":false,"hidden":false,"mode":"md"},{"id":1160,"value":"consumption_samples_stdev = 82.18","pinned":false,"hidden":false,"mode":"ojs"},{"id":1324,"value":"We can then generate a collection of observations (These observations are fictitious, all the mean, but it doesn't matter because bayesian inference only uses the sum and length of this array, see [Appendix C](#appendixc)).","pinned":false,"hidden":false,"mode":"md"},{"id":1254,"value":"consumption_observations = Array(consumption_sample_count).fill(Math.log(consumption_per_person_per_year_usd))","pinned":true,"hidden":false,"mode":"ojs"},{"id":1330,"value":"I then choose an appropriate prior. This is somewhat arbitrary and should not impact the result, as we have enough samples for the prior to be not considered very strongly. It comes from guessing the mean consumption to be somewhere from `100 to 500` on squiggle:","pinned":false,"hidden":false,"mode":"md"},{"id":1167,"value":"consumption_prior = ({mean: 5.409889142205142, stdev: 0.48923438720106327 })","pinned":false,"hidden":false,"mode":"ojs"},{"id":1366,"value":"cell(`lognormal($consumption_prior.mean, $consumption_prior.stdev)`, [], {jsImports: {consumption_prior: consumption_prior}, tickFormat: \"$,f\", title: \"Baseline consumption prior (USD)\"})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1334,"value":"We then calculate the posterior based on the prior and observations^{[5](#footnotetosample)}.","pinned":false,"hidden":false,"mode":"md"},{"id":1170,"value":"consumption_posterior = normal_normal_bayes_update(consumption_prior, consumption_observations, Math.log(consumption_samples_stdev))","pinned":false,"hidden":false,"mode":"ojs"},{"id":228,"value":"viewof baseline_consumption = cell(`baseline_consumption = SampleSet.fromDist(lognormal($consumption_posterior.mean, $consumption_posterior.stdev))`, [], {jsImports: {consumption_posterior: consumption_posterior}})","pinned":false,"hidden":false,"mode":"ojs"},{"id":574,"value":"household_size_data = FileAttachment(\"household size analysis - Family size by household.csv\").csv()","pinned":false,"hidden":false,"mode":"ojs"},{"id":582,"value":"household_sizes = household_size_data.map(x => parseFloat(x[\"Total household size\"])).filter(x => x !== 0)","pinned":true,"hidden":false,"mode":"ojs"},{"id":1686,"value":"FrequencyChart(household_sizes)","pinned":true,"hidden":false,"mode":"ojs"},{"id":1114,"value":"household_sizes_prior = ({shape: 3, scale: 1.5})","pinned":false,"hidden":false,"mode":"ojs"},{"id":2823,"value":"cell(`gamma($prior.shape, $prior.scale)`, undefined, {jsImports: {prior: household_sizes_prior}})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1125,"value":"household_sizes_posterior = poisson_gamma_bayes_update(household_sizes_prior, household_sizes)","pinned":true,"hidden":false,"mode":"ojs"},{"id":353,"value":"viewof household_size = cell(`household_size = gamma($household_sizes_posterior.shape, $household_sizes_posterior.scale)`, [], {jsImports: {household_sizes_posterior: household_sizes_posterior}})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1851,"value":"## Problems and room for improvement with this model\nHere I go through some larger changes to the model. Most of these models are here to ensure that the model remains internally consistent.","pinned":false,"hidden":false,"mode":"md"},{"id":3140,"value":"givedirectly_params = givewell_params","pinned":false,"hidden":false,"mode":"ojs"},{"id":5373,"value":"### Continuous Present Value\nWe use the continuous version of present value, with the formula detailed above. This is because I think it models the situation better.","pinned":false,"hidden":false,"mode":"md"},{"id":5378,"value":"viewof continuous_present_value = Inputs.toggle({label: \"Use Continuous Present Value\"})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1830,"value":"$3a","pinned":false,"hidden":false,"mode":"md"},{"id":1810,"value":"I'm only about 75% sure that this change is appropriate. If you would like to use an isoelastic utility function with ${tex`\\eta = 1.59`}, please tick the following:","pinned":false,"hidden":false,"mode":"md"},{"id":1814,"value":"viewof isoelastic_utility = Inputs.toggle({label: \"Use isoelastic utility function\"})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1837,"value":"### Other changes not added but desired","pinned":false,"hidden":false,"mode":"md"},{"id":1771,"value":"$3b","pinned":false,"hidden":false,"mode":"md"},{"id":934,"value":"## Minor changes made to GiveWell's model\nI made a number of changes in this model to correct what I see as a collection of errors made in the model design. None of these errors change the results by any large degree. They are mostly stylistic and I doubt are contraversial. With these changes, the model should still remain mostly comparable to the original.\n\n${list(minor_changes)}","pinned":false,"hidden":false,"mode":"md"},{"id":2961,"value":"minor_changes = [`We use the continuous version of present value, with the formula detailed above. This is because I think it models the situation better.`, `We multiply, rather than add the adjustment percentages. This is because [you can't add percentages](https://towardsdatascience.com/most-people-screw-up-multiple-percent-changes-heres-how-to-do-get-them-right-b86bd6ef4b72)`, `We the [implicit parameter for duration of initial consumption](https://forum.effectivealtruism.org/posts/WmQwtYEajNDuPdyZx/type-checking-givewell-s-givedirectly-cost-effective#Results) explicit. *This makes no changes to the result*.`, `We remove the donation size parameter. The choice of this parameter is entirely arbitrary and does not impact the result at all. It is simply multiplied then divided out to calculate the doublings of consumption per dollar. On discussion with David Reinstein, I thought it better to remove it.`]","pinned":true,"hidden":false,"mode":"ojs"},{"id":822,"value":"## Acknowledgements\nBoth Squiggle and this work was done with help and funding from the [Quantified Uncertainty Research Institute](https://quantifieduncertainty.org/). I would like to thank Ozzie Gooen for his feedback, and Nuño Sempere for contributing their feedback and estimation expertise to this project. I'd like to thank the rest of the Squiggle development team: Ozzie Gooen, Nuño Sempere and Quinn Dougerty. I'd further like to thank David Reinstein for his feedback and interest on this project.\n\n![Quantified Uncertainty Research Institute](https://super-static-assets.s3.amazonaws.com/09bb1362-5e3f-4724-8ffd-f3235f67356f/images/6151ac3e-aed7-44c7-9827-399fe6e9222b.png)","pinned":false,"hidden":false,"mode":"md"},{"id":688,"value":"## Footnotes","pinned":false,"hidden":false,"mode":"md"},{"id":709,"value":"footnote = (referencingCellName, number, text) => {\n return md`

^ ${text}

`;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1038,"value":"footnoteto = footnote(\"cell-177\", 1, `The \"to\" syntax here is [syntax sugar for a lognormal distribution with a 90% confidence interval between the two numbers](https://www.squiggle-language.com/docs/Features/Functions#lognormal-distribution). This is borrowed from guesstimate.`)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1088,"value":"footnotetransfer = footnote(\"cell-177\", 2, `I feel like there's almost certainly a better way to do this than this estimate. We have all the spending data that we could possibly use, and we could probably do some form of bayesian inference to get the actual value of this parameter. I'm just not that sure on how to do that.`)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1695,"value":"footnotemean = footnote(\"cell-1297\", 3, `This is mildly problematic, because it would be better if we were to use the entire distribution rather than the point estimate of the mean. However Squiggle does not [yet support distributions as parameters to other distributions](https://github.com/quantified-uncertainty/squiggle/discussions/482). However, due to the household size having a small variance, I doubt this would have a very large impact on the final result.`)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1406,"value":"footnoteknownvariance = footnote(\"cell-1318\", 4, `\nIn this bayesian inference, I assume we know the variance. In truth, I don't think we actually know the variance, because I'm getting my variance value from the observations. I'm just not currently good enough at bayesian inference to do this properly. However, I doubt it would make much of a chance on the final result.`)","pinned":false,"hidden":false,"mode":"ojs"},{"id":722,"value":"footnotetosample = footnote(\"cell-220\", 5, `In Squiggle, under allmost all situations, distributions are assumed to be indipendent. This becomes an issue, for instance when you are measing log increase in consumption: \\`log(baseline_consumption + amount_consumed) - log(baseline_consumption) \\`.\n\nHere, the distributions that are being subtracted from each other are not indipendent, because they both rely on the same variable \\`baseline_consumption\\`. If they were, you would get nonsense results such as having probability mass below 0.\n\nHowever, sample sets do not have this property, and allow you to do operations on distributions that are not indipendent. This is why I convert \\`baseline_consumption\\` to a sample set\n\nTo take a look at the different types of distributions available in Squiggle, [check out our documentation](https://www.squiggle-language.com/docs/Discussions/Three-Formats-Of-Distributions)`)","pinned":false,"hidden":false,"mode":"ojs"},{"id":840,"value":"footnotecontinuouspv = footnote(\"cell-485\", 6, `Another reason that I prefer the continuous format is because I love Dimensional Analysis (checking units of calcualtions), and the continuous version is [much easier to solve units for](https://forum.effectivealtruism.org/posts/WmQwtYEajNDuPdyZx/type-checking-givewell-s-givedirectly-cost-effective#Problems_and_caveats).`)","pinned":false,"hidden":false,"mode":"ojs"},{"id":753,"value":"footnotesellinginvestments = footnote(\"cell-503\",7,`I'm not sold that this is an important consideration and could complicate the model. Removal of this consideration would be a change I would make.`)","pinned":false,"hidden":false,"mode":"ojs"},{"id":551,"value":"## Appendix A, Squiggle features that could improve this model.","pinned":false,"hidden":false,"mode":"md"},{"id":138,"value":"Current rooms for improvement, particularly within Squiggle development:\n - The performance is pretty slow at this point. It would be valuable to improve it.\n - Allowing parameters of distribution to be themselves distributions. Would be useful for bayesian inference of household sizes above","pinned":false,"hidden":false,"mode":"md"},{"id":1187,"value":"$3c","pinned":false,"hidden":false,"mode":"ojs"},{"id":2941,"value":"import {normal_normal_bayes_update, poisson_gamma_bayes_update} from \"@hazelfire/bayesian-updating-with-conjugate-priors\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":1462,"value":"### Utility Functions","pinned":false,"hidden":false,"mode":"md"},{"id":1119,"value":"sum = x => x.reduce((a, b) => a + b)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1456,"value":"formatFloat = x => new Intl.NumberFormat('en-US', {maximumSignificantDigits: 3}).format(x)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1458,"value":"formatCurrency = x => new Intl.NumberFormat('en-US', { style: 'currency', currency: 'USD' }).format(x)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1620,"value":"import { BarChart } from \"@d3/bar-chart\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":1514,"value":"import {toc} from \"@harrystevens/toc\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":1673,"value":"freq = data => data.reduce((acc, d) => {\n if (!acc[d]) {\n acc[d] = 0;\n }\n acc[d]++;\n return acc\n }, {});","pinned":false,"hidden":false,"mode":"ojs"},{"id":1675,"value":"FrequencyChart = data => BarChart(Object.entries(freq(data)), {\n x: d => d[0],\n y: d => d[1],\n xDomain: Object.keys(freq(data)), // sort by descending frequency\n yFormat: \"g\",\n yLabel: \"↑ Count\",\n width,\n height: 300,\n color: \"steelblue\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":2964,"value":"list = (items) => items.map((x, i) => `${i + 1}. ${x}`).join(\"\\n\")","pinned":true,"hidden":false,"mode":"ojs"},{"id":1996,"value":"$$ = tex.options({trust: true, displayMode: true, macros: {\"\\\\param\": context => { \n const args = context.consumeArgs(1);\n const name = args[0].reverse().map(x => x.text).join(\"\")\n const url = name.replaceAll(\" \", \"_\")\n return `\\\\textit{\\\\href{\\\\#${url}}{${name}}}`}, \"\\\\ref\": context => { \n const args = context.consumeArgs(1);\n const name = args[0].reverse().map(x => x.text).join(\"\")\n const url = name.replaceAll(\" \", \"_\")\n return `\\\\text{\\\\href{\\\\#${url}}{${name}}}` }\n}})","pinned":true,"hidden":false,"mode":"ojs"},{"id":3036,"value":"gw_gd_cea = FileAttachment(\"gw_gd_cea.csv\").csv()","pinned":true,"hidden":false,"mode":"ojs"},{"id":3111,"value":"param_imports = {\n let x = {}\n gw_gd_cea.forEach(entry => {\n const parameter_name = entry[\"Cash Transfers - GiveDirectly\"].replace(\"\\\"\", \"\")\n let value = entry[\"Overall\"].replace(\",\",\"\")\n if(value === \"Not specified\"){\n value = 0\n }\n if(!Number.isNaN(parseFloat(value))){\n x[parameter_name] = parseFloat(value)\n }\n\n \n })\n return x;\n}\n ","pinned":false,"hidden":false,"mode":"ojs"},{"id":3033,"value":"$3d","pinned":true,"hidden":false,"mode":"ojs"},{"id":3063,"value":"sensitivity = (func, params, b) => {\n let result = run(`cov(x,y) = mean(x * y) - mean(x) * mean(y)\ncorr(x,y) = cov(x, y) / stdev(x) / stdev(y)\nr_squared(x, y) = corr(x, y) ^ 2\nall_to_sample_set(x) = Dict.fromList(map(Dict.keys(x), {|param| [param, SampleSet.fromDist(x[param])]}))\nsensitivity(f, params) = {\n ss_params = all_to_sample_set(params)\n result = f(ss_params)\n Dict.fromList(map(Dict.keys(ss_params), {|param| [param, r_squared(ss_params[param], result)]}))\n}\nsensitivity(${func}, ${params})`, b).value.value.entries()\n return Plot.plot({\n marks: [\n Plot.barX(result, {x: d => d[1].value, y: d => d[0], sort: {y: \"x\", reverse: true}}),\n Plot.ruleX([0])\n ],\n marginLeft: 200\n })\n}","pinned":true,"hidden":false,"mode":"ojs"}],"files":"$W3e","stdlib":"1","titling":"auto"}