2e:null 2f:["$","title",null,{"children":"GiveWell GiveDirectly Cost Effectiveness Analysis | Nuño Sempere | Observable"}] 34:T4c7,## Motivation for this work This work was originally motivated by wanting to see what can be done to improve GiveWell's Cost Effectiveness Analysis. The most obvious addition was to quantify uncertainty. There's a lot of value in quantifying uncertainty for cost effectiveness, such as improving conversation about uncertainty and forecasting. My favourite is the ability to calculate [value of information](https://en.wikipedia.org/wiki/Value_of_information). We could work out when it would be better to learn about an intervention rather that investing in an intervention. Calculating the value of information is out of the scope of this analysis. This serves a dual purpose of demonstrating Squiggle. I'm very excited for the possibilities available with Squiggle and Squiggle notebooks. Squiggle notebooks allow you to quantify uncertainty, but also allow you to write and explain the assumptions of the model in text form. They include links, images, tables, and can pull data from other locations. It's also much easier to use and learn than a full probabilistic programming language. I hope that others will be inspired to create their own Squiggle notebooks for their work, and enjoy the process as much as I did.35:T5bc,## Notes on the implementation To see the code that was used to create this notebook, hover over a cell and click the caret to the left. The text cells will mostly show you markdown, but the squiggle cells will show you JavaScript. Observable is a JavaScript notebook platorm, and we've used it to embed squiggle notebooks inside. You can change the text in the cells to see how it would change the model. Please note that currently this isn't that performant. [We are working on improving the performance to make a better experience](https://github.com/quantified-uncertainty/squiggle/issues/465). Changing the text does not change the model or anyone else but yourself, and refreshing the browser or even changing some other cells will remove your changes. If you wish to persist your changes, you can edit the string passed in by the JavaScript code. If you wish, you can also fork this model and suggest changes that I may merge into this one. This is all supported by observable. You'll note that there are cells in this notebook that show graphs and cells that don't and just declare variables. The former we call "cells" and the latter we call "partials". "partials" can declare variables but not return values, and "cells" vise versa. We hope that in the future we'll have an implementation where the one cell does both soon. If you wish to learn more about writing your own notebooks, check out [our tutorial](https://observablehq.com/@hazelfire/squiggle).36:T4c7,The recipient is then modelled to invest the other portion of their grant. The return on investment from the grant was chosen by GiveWell analysis, here's a discussion from GiveWell analysists: > **Notes from former GiveWell Research Analyst Chris Smith**: I would guess that roofs are an unusually good investment item. I expect that most other assets GiveDirectly's recipients invest in offer lower returns for fewer years. > > It's also important to account for the fact that many items counting as assets (for the sake of the suggested 39% input for "Percentage of transfers invested") aren't interest-bearing (at least in the conventional sense). For example, many people use transfers to buy furniture. > **Notes from GiveWell Senior Fellow Caitlin McGugan**: > Taking the conservative side estimate of the return to iron roofs. I agree with Chris that there’s probably a mix of investments, some of which is in durable consumption goods that don’t earn a return, bringing the average rate of return down. But I also think this is partially compensated by people selecting into non-roof investments (such as livestock) because they believe they’ll be more successful than average in that type of investment. 37:T516,The investment period runs over a duration. This duration is a very uncertain value because follow up studies are yet to be performed by GiveDirectly. Here are some discussions on the value by GiveWell staff: > **Notes on this input from GiveWell Senior Fellow Caitlin McGugan**: > This input is very uncertain as we lack sufficiently long follow-up evidence for GiveDirectly recipients. I'm choosing a value that I think is relatively optimistic but somewhat below the expected lifespan of a roof or a goat. > On the one hand, I think the transfers and some investments made using them can help buffer households from income shocks, prolonging the time until they’re forced to consider selling assets. On the other hand, I think negative income shocks are frequent in these populations, and there’s a good chance that households won’t be able to hang onto their investment over the long term. The balance of evidence I'm aware of from other types of transfer programs suggests that it's challenging for households to maintain asset investments over the very long term. > **Notes on this input from GiveWell Senior Advisor Michael Eddy**: Recent evidence (from Uganda, Ethiopia & Nairobi) makes me generally pessimistic on the long-term (> 5 year) impacts of GiveDirectly-style cash transfers.38:T470,# Appendix B, Changes made to GiveWell's model I made a number of changes in this model to correct what I see as a collection of errors made in the model design. None of these errors change the results by any large degree. 1. I prefer to use the continuous version of present value, with the formula detailed above. This is because I think it models the situation better. 2. I prefer to multiply, rather than add the adjustment percentages. This is because [you can't add percentages](https://towardsdatascience.com/most-people-screw-up-multiple-percent-changes-heres-how-to-do-get-them-right-b86bd6ef4b72) 3. I make the [implicit parameter duration of initial consumption](https://forum.effectivealtruism.org/posts/WmQwtYEajNDuPdyZx/type-checking-givewell-s-givedirectly-cost-effective#Results) explicit. 4. GiveWell's definition of value is the doubling of consumption, but they calculate the increase in consumption in natural logarithms. This means they need to end the analysis by multiplying by 1.44 or ${tex`\log_2(e)`}. I decided to change the ${tex`\log_e(x)`} to ${tex`\log_2(x)`} so that I don't need that conversion factor.39:[{"id":"f1018680e8aebbe90f5462669245558e829749a35c8e590344ee54a197e9f7acd41dbe383c837a9b6033d9471ca57fbe9721f2c8f02b598763982e4bf714f8ef","href":"https://static.observableusercontent.com/files/f1018680e8aebbe90f5462669245558e829749a35c8e590344ee54a197e9f7acd41dbe383c837a9b6033d9471ca57fbe9721f2c8f02b598763982e4bf714f8ef","name":"Copy of GiveWell-s analysis of GiveDirectly financial summary through February 2018 - Sam-s Summary.csv","type":"text/csv","lastModified":1651603557648.181,"size":1022},{"id":"8ad0773214d8e7f14ef0a9adcc4679f7e6e1bd9a5f9a6037dbb852d52a33c0f68537840d1c83df3fd2ed396a66edb1bd472f694308052c3e50c3c04ed354c198","href":"https://static.observableusercontent.com/files/8ad0773214d8e7f14ef0a9adcc4679f7e6e1bd9a5f9a6037dbb852d52a33c0f68537840d1c83df3fd2ed396a66edb1bd472f694308052c3e50c3c04ed354c198","name":"household size analysis - Family size by household.csv","type":"text/csv","lastModified":1651603557648.181,"size":1634}] 32:{"id":"8dc2c8c2b62b5970","slug":"givewell-givedirectly-cost-effectiveness-analysis","title":"GiveWell GiveDirectly Cost Effectiveness Analysis","thumbnail":"cfaee074ab563d5661ce64fa519f6c1370ada6d22d118cbf0cff8141f1437f83","access_level":"public","create_time":"$D2022-05-03T18:45:57.648Z","update_time":"$D2022-05-03T19:15:56.985Z","view_time":null,"first_public_time":"$D2022-05-03T19:15:10.721Z","has_unpublished_edits":null,"fork":{"id":"38cfea3f6eaa5311","slug":"givewells-givedirectly-cost-effectiveness-analysis","title":"Quantifying Uncertainty in GiveWell's GiveDirectly Cost Effectiveness Analysis","owner":{"id":"1fff94fffef46139","login":"hazelfire","name":"Sam Nolan"}},"stars":0,"starred":false,"owner":{"type":"team","id":"07a813a05afbeeaf","login":"nunosempere","name":"Nuño Sempere","avatar_url":"https://avatars-next.observableusercontent.com/512eae7d32fc25bba414910bc3c00fac559004bc68c3e14fa07fa7c6544b630f","tier":"starter_2024"},"version":1100,"publish_version":1100,"role":null,"authored":false,"authors":[{"type":"individual","id":"cee02f8e9acb9aee","login":"nunosempere","name":"Nuño Sempere","avatar_url":"https://avatars-next.observableusercontent.com/bc840e103afe5117def6829a8a8476725da032e7bb7cc3cb8c49257be439a233","tier":null}],"members":[{"type":"individual","id":"cee02f8e9acb9aee","login":"nunosempere","name":"Nuño Sempere","avatar_url":"https://avatars-next.observableusercontent.com/bc840e103afe5117def6829a8a8476725da032e7bb7cc3cb8c49257be439a233","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":null,"type":"notebook","body":{"title":"GiveWell GiveDirectly Cost Effectiveness Analysis","cells":[{"id":124,"value":"# GiveWell GiveDirectly Cost Effectiveness Analysis","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":407,"value":"## Introduction","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":58,"value":"This is an example of a squiggle model for [GiveWell's GiveDirectly Cost Effectiveness Analysis](https://docs.google.com/spreadsheets/d/1Kq6iHSQFr3eRz1p9KclHuJTQiaJYkpViOyneSd9KCJc/edit#gid=1680005064). This is meant to be an improvement over the original spreadsheet by providing quantified uncertainty, as well as giving context on where all the variable values come from and the major uncertainties.\n\nThe notebook is also interactive, allowing you to generate evaluations based on your own preferred values for parameters, allowing you to model, change or even suggest changes.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":641,"value":"## What's Squiggle?\n[Squiggle](https://squiggle-language.com/) is an estimation language for forecasters and evaluators. This notebook uses squiggle to represent the uncertainty with certain parameters. Squiggle is in pre-alpha stage, and comes with a collection of bugs and missing features if you were to use it yourself. However, we are speeding towards a usable version! This here represents an example use of squiggle to quantify uncertainty in GiveWell's CEA.\n\nIf you want to use Squiggle, I recommend checking out the [playground](https://www.squiggle-language.com/playground/). For observable support, check out my [demo Squiggle notebook](https://observablehq.com/@hazelfire/squiggle). Also feel free to directly get in contact with me.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":646,"value":"$34","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":674,"value":"$35","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":324,"value":"import {cell, partial} from '@hazelfire/squiggle'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":651,"value":"# GiveDirectly model","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":413,"value":"## Funding Distribution","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":169,"value":"GiveWell models start with a donation size in USD. This donation size is entirely arbitrary, and could be $1 to $1,000,000,000. The calculations should give the same amount of impact per dollar. The amount given in the GiveWell spreadsheet is $10,000. I let you chose a donation size:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":172,"value":"viewof donation_size = Inputs.number({value: 10000, step: 1, label: \"Donation Size\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":606,"value":"donation_size","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":177,"value":"The proportion of funding available is the funding that ends up in the hands of grantees, after any required administration costs. This value is given a point value of 83% by GiveWell, and is calculated from [an analysis of GiveDirectly spending in February 2018](https://docs.google.com/spreadsheets/d/1L03SQuAeRRfjyuxy20QIJByOx6PEzyJ-x4edz2fSiQ4/edit#gid=537899494). I use a copy of this spreadsheet to construct a distribution of possible spending percentages based on a period of time from August 2016 to February 2018. You can see/download this copy by clicking on the attachments button on the right of this notebook.^{[1](#footnote1)}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":592,"value":"spending_data = FileAttachment(\"Copy of GiveWell-s analysis of GiveDirectly financial summary through February 2018 - Sam-s Summary.csv\").csv()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":968,"value":"sum= arr => arr.reduce((a,b) => a+b,0)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":967,"value":"mean_transfer_efficiency = {\n let totalMoneyTransfered = sum(\n spending_data.map((transfer) => parseFloat(transfer[\"Money to Transfers\"].replaceAll(\",\", \"\")))\n );\n let totalSpending = sum(\n spending_data.map((transfer) => parseFloat(transfer[\"Total spending\"].replaceAll(\",\", \"\")))\n );\n let efficiency = totalMoneyTransfered / totalSpending;\n\n return efficiency;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1029,"value":"nuno_guess_transfer_efficency_going_forward = \"0.75 to 0.9\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1068,"value":"cell(nuno_guess_transfer_efficency_going_forward)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1048,"value":"transfer_efficiency = nuno_guess_transfer_efficency_going_forward","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":594,"value":"transfer_percentages = spending_data.map(x => parseFloat(x[\"Percentage on Transfers\"]))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":179,"value":"viewof bindings1 = partial(`proportion_of_funding_available = ${transfer_efficiency}\ntotal_funding_available = $donation_size * proportion_of_funding_available`, {}, {donation_size: donation_size})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":338,"value":"cell(`total_funding_available`, bindings1)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":188,"value":"A second important measure is how much money goes to a household. A GiveDirectly grant is usually [given to a household](https://www.givewell.org/charities/give-directly#Overview). GiveDirectly aims to give different amounts to different demographics based on inflation and other considerations. GiveWell gives a point estimate of $1000 based on talking with GiveDirectly:\n\n> [Transfer sizes for the standard lump sum projects: Kenya: $1,085, Rwanda: $970, Uganda: $963.](https://www.givewell.org/charities/give-directly/supplementary-information#footnote2_0h0watm)\n>\n> February 20, 2018, [Joe Huston](https://www.givedirectly.org/author/joe-huston/) GiveDirectly CFO","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":194,"value":"viewof size_of_transfer = partial(`size_of_transfer = 925 to 1125`, bindings1)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":363,"value":"cell(\"size_of_transfer\", size_of_transfer)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":349,"value":"The money given to a participant is shared among a household. We measure how much good we do for each person, so we'll need the average household size. A point estimate of 4.7 is provided by GiveWell from their [analysis on household size](https://files.givewell.org/files/DWDA%202009/GiveDirectly/household%20size%20analysis.xls) which uses data from the data from the Siaya 2012 census. I made a copy of this analysis and used it to get the distribution of possible household sizes.^{[2](#footnote2)}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":574,"value":"household_size_data = FileAttachment(\"household size analysis - Family size by household.csv\").csv()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":582,"value":"household_sizes = household_size_data.map(x => parseFloat(x[\"Total household size\"])).filter(x => x !== 0)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1086,"value":"[Math.min(...household_sizes), Math.max(...household_sizes)] // Just for reference","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":353,"value":"viewof household_size = partial(`household_size = fromSamples($household_sizes)\nsize_of_transfer_per_person = size_of_transfer / household_size`, size_of_transfer, {household_sizes: household_sizes})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":367,"value":"cell(`[household_size, size_of_transfer_per_person]`, household_size)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":198,"value":"Recipients can choose to spend the money now or invest it in something that gives them returns in the future. The point estimate for GiveWell's estimate is that participants invest 39% of the grant. An investment in this case is simply a purchase that has a pay off over a period of time. This figure is retrieved from [Haushofer & Shapiro 2013](https://files.givewell.org/files/DWDA%202009/Interventions/Cash%20Transfers/haushofer_shapiro_uct_2013.11.16.pdf).\n> \"Households invest part of the transfers: the value of non-land assets increased by USD 279 on average; this represents 58 percent of the control group mean USD 478, and 39 percent of the average transfer.\"\n\nThe actual distribution parameters chosen are also somewhat arbitrary, and could be more conservative than it should. I could improve this by using the original dataset used in the study if I could get access to it. (Nuno)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":420,"value":"## Recipient usage of funds","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":423,"value":"### Initial consumption","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":206,"value":"viewof amount_invested = partial(`portion_invested = beta(32, 50)\namount_invested = portion_invested * size_of_transfer_per_person`, household_size)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1097,"value":"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":385,"value":"cell(`{portion: portion_invested, amount: amount_invested}`, amount_invested)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":381,"value":"Everything that is not invested is consumed over a period of 1 year.\nIn GiveWell's model, the fact that it is [consumed over 1 year is implicit](https://forum.effectivealtruism.org/posts/WmQwtYEajNDuPdyZx/type-checking-givewell-s-givedirectly-cost-effective). Although I would prefer to make it explicit, I keep it implicit for simplicity's sake.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1099,"value":"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":375,"value":"viewof amount_consumed = partial(`portion_consumed = 1 - portion_invested\namount_consumed = portion_consumed * size_of_transfer_per_person`, amount_invested)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":388,"value":"cell(`{portion: portion_consumed, amount: amount_consumed}`, amount_consumed)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":220,"value":"We measure increase in welfare by doubling of consumption. The baseline consumption for recipients is given as 285.92 USD per year by GiveWell. To quote GiveWell:\n\n> Taken from [Haushofer and Shapiro 2013](https://files.givewell.org/files/DWDA%202009/Interventions/Cash%20Transfers/haushofer_shapiro_uct_2013.11.16.pdf), Pg 49. Non-durable expenditure (USD) in the control group is measured as 157.40 a month. This is multiplied by 12 to get an annual estimated, adjusted for purchasing power parity, and divided by average household size. See [calculations](https://files.givewell.org/files/DWDA%202009/Interventions/GiveWell_cost-effectiveness_analysis_2016.xlsx) [Parameters > Cash] for more details.\n\nThe actual distribution here is considered a bit of a guess, and a wide one at that. Possible something Nuno could help with. We also need to use `toSampleSet` to ensure that a calculation lower down works^{[3](#footnote3)}.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":228,"value":"viewof baseline_bindings = partial(`baseline_consumption = toSampleSet(200 to 350)`, amount_consumed)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":333,"value":"cell(`baseline_consumption`, baseline_bindings)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":230,"value":"The initial consumption period lasts the first year. This parameter is implicit within GiveWell's analysis. Here it's made explicit. From this we can calculate how much doubling of consumption we create.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":239,"value":"viewof log_increase_in_initial_consumption = partial(`duration_of_initial_consumption = 1\nlog_increase_in_initial_consumption = (log(amount_consumed / duration_of_initial_consumption + baseline_consumption, 2) - log(baseline_consumption, 2)) * duration_of_initial_consumption`, baseline_bindings)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":448,"value":"cell(`log_increase_in_initial_consumption`, log_increase_in_initial_consumption)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":430,"value":"### Recipient Investments","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":438,"value":"$36","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":258,"value":"viewof log_increase_in_consumption_from_investment = partial(`return_on_investment = 0.08 to 0.12\nincrease_in_consumption_from_investments = return_on_investment * amount_invested\nlog_increase_in_consumption_from_investment = log(increase_in_consumption_from_investments + baseline_consumption, 2) - log(baseline_consumption, 2)`, log_increase_in_initial_consumption)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":330,"value":"cell(`log_increase_in_consumption_from_investment`, log_increase_in_consumption_from_investment)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":454,"value":"$37","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":281,"value":"viewof investment_duration = partial(`investment_duration = 8 to 12`, log_increase_in_consumption_from_investment)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":479,"value":"GiveWell has a [full discussion of their discount rate elsewhere](https://docs.google.com/document/d/1g2tf7Fu8-iG_8TgnJkEx9G5TlEyrcv7PXPokmPZaUMc/edit?usp=sharing). Note that the discount rate is not because there is an inherent value on getting utility now, but is made from three components:\n- Uncertainty in the future\n- Increase in monetary consumption due to other effects, making marginal log increases in consumption later more difficult.\n- Increase in non-monetary consumption, again making marginal log increases in consumption more difficult.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":467,"value":"viewof discount_rate = partial(`discount_rate = beta(10, 200)`, investment_duration)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":741,"value":"cell(`discount_rate`, discount_rate)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":485,"value":"This is a continuous present value formula, it is different from the one used by GiveWell. I prefer it because I think GiveDirectly investments are probably not well modelled by discrete payouts^{[4](#footnote4)}. It comes from the following formula for present value:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":487,"value":"p \\int_0^t (1 + d)^{-x} dx = p\\frac{1 - (1 + d)^{-t}}{\\log_e(1 + d)}","pinned":false,"hidden":false,"mode":"tex","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":561,"value":"Where is p the payment, t is the duration, d is the discount_rate ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":471,"value":"viewof pv_excluding_last_year = partial(`pv(payment, discount, duration) = payment * (1 - (1 + discount) ^ (-duration)) / log(discount_rate + 1)\npresent_value_excluding_last_year = pv(log_increase_in_consumption_from_investment, discount_rate, investment_duration - 1)`, discount_rate)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":761,"value":"cell(`present_value_excluding_last_year`, pv_excluding_last_year)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":503,"value":"GiveWell then considers the benefits from reselling investments. Comments from GiveWell:\n> In some cases, items purchased with cash transfers might be resold when they cease providing returns. We are extremely uncertain about the appropriate value for this parameter. It's our best guess that most investment items will not retain much of their original value. We have not prioritized a thorough investigation into possible values for this parameter.^{[5](#footnote5)}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":290,"value":"viewof pv_last_year = partial(`percent_of_investment_returned = 0.15 to 0.25\npv_consumption_last_year = (log(baseline_consumption + amount_invested * (return_on_investment + percent_of_investment_returned)) - log(baseline_consumption)) / (1 + discount_rate)^investment_duration`, pv_excluding_last_year);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":758,"value":"cell(`pv_consumption_last_year`, pv_last_year)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":508,"value":"We can then add the three different sources of value together to create the total increase in pv from cash transfers.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":298,"value":"viewof total_pv = partial(`total_pv_of_cash_transfer = pv_consumption_last_year + present_value_excluding_last_year + log_increase_in_initial_consumption`, pv_last_year)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":526,"value":"cell(`total_pv_of_cash_transfer`, total_pv)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":545,"value":"## Negative Spoiler and Total Value","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":529,"value":"The negative spoiler represents possible negative externalities that come from direct cash transfers. GiveWell has an [analysis of negative spoiler from cash transfers](https://docs.google.com/document/d/1C4nX3LWM-TeNMFxmNAKRbi4vtt_4ZaLoIGZzlaHmvuI/edit#).","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":512,"value":"viewof negative_spoiler = partial(`discount_negative_spoiler = 0.93 to 0.97\nvalue_discounting_spoiler = discount_negative_spoiler * total_pv_of_cash_transfer`, total_pv)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":539,"value":"Finally, we total up the amount of value created for each household/transfer","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":515,"value":"viewof total_value = partial(`consumption_increase_per_household = value_discounting_spoiler * household_size\namount_of_transfers_made = total_funding_available / size_of_transfer\ntotal_increase_in_ln_consumption = amount_of_transfers_made * consumption_increase_per_household\nvalue_per_dollar = total_increase_in_ln_consumption / $donation_size`, negative_spoiler)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":326,"value":"cell(`total_increase_in_ln_consumption`, total_value)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":803,"value":"cell(`value_per_dollar`, total_value)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":849,"value":"## Charity level adjustments\nThe main part of the analysis is done. In the next section, we go through possible positive and negative adjustments to the impact based on other considerations that are not inside the model.\nConsider all numbers here to be percentages that get multiplied by the final result, like an adjustment of 1 means 100%, and is actually no adjustment.\nSimilar adjustments are considered for all investments, as of such a number of these adjustments are simply not applicable for GiveDirectly.\n### Risks of wastage\nThe adjustments are defined by GiveWell as follows:\n> **double_treatment** The program is delivered to people who don't benefit or minimally benefit because they have received the program from another source. *Not applicable for GiveDirectly*\n\n> **ineffective_goods** Pills, LLINs, etc. are poorly made, counterfeit, or otherwise largely or fully ineffective at achieving the expected outcome. *Not applicable for GiveDirectly*","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":852,"value":"viewof risk_of_wastage = partial(`double_treatment = 1\nineffective_goods = 1\ncash_doesnt_reach_participants = 1.03\ntotal_adjustment_for_risk_of_wastage = double_treatment * ineffective_goods * cash_doesnt_reach_participants`, total_value)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":859,"value":"### Quality of monitoring and evaluation","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":862,"value":"viewof quality_of_monitoring = partial(`misappropriation = 1\nfalse_monitoring = 1\ntotal_adjustment_of_monitoring_and_evaluation = misappropriation * false_monitoring`, risk_of_wastage)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":868,"value":"### Confidence of funds being used for intended purpose","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":870,"value":"viewof cofidence_of_funds = partial(`change_of_priorities = 1\nnon_funding_bottlenecks = 1\nwithin_org_fungability = 1.05\ntotal_confidence_of_funds = change_of_priorities * non_funding_bottlenecks * within_org_fungability`, quality_of_monitoring)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":898,"value":"### Total value after adjustment","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":886,"value":"viewof value_after_charity_adjustments = partial(`total_charity_adjustment = total_adjustment_for_risk_of_wastage * total_adjustment_of_monitoring_and_evaluation * total_confidence_of_funds\nvalue_per_dollar_after_charity_adjustments = value_per_dollar / total_charity_adjustment`, cofidence_of_funds)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":894,"value":"cell(`value_per_dollar_after_charity_adjustments`, value_after_charity_adjustments)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":909,"value":"## Intervention Level Adjustments","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":913,"value":"viewof intervention_level_adjustments = partial(`changes_in_ppp = 0.93\ndevelopmental_effects = 1.05\nreduced_morbidity = 1.04\nchild_mortality_effects = 1.04\n\ntotal_adjustment_factor = changes_in_ppp * developmental_effects * reduced_morbidity * child_mortality_effects`, value_after_charity_adjustments)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":916,"value":"viewof final_value_per_dollar = cell(`value_per_dollar_after_all_adjustments = total_adjustment_factor * value_per_dollar_after_charity_adjustments\nvalue_per_dollar_after_all_adjustments`, intervention_level_adjustments)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":922,"value":"md`The final value per dollar has a mean of **${final_value_per_dollar.value.mean().value.toFixed(5)}** and a 95% confidence interval of **${final_value_per_dollar.value.inv(0.05).value.toFixed(5)}** and **${final_value_per_dollar.value.inv(0.95).value.toFixed(5)}** value / USD.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":777,"value":"## Discussion\nI had a lot of fun writing this Squiggle Notebook. It offers a large collection of advantages over other alternatives:\n- Ability to write detail inbetween cells\n- External links\n- Ability to quantify uncertainty\n- Ability to create calculators and interactive analysis\n\nIn the future, I would love to continue this work in quantifying uncertainty for the rest of the GiveWell models, or possibly bringing back quantification for the [ACE models](https://animalcharityevaluators.org/charity-reviews/evaluating-charities/evaluation-criteria/#ce).","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":822,"value":"## Acknowledgements\nBoth Squiggle and this work was done with help and funding from the [Quantified Uncertainty Research Institute](https://quantifieduncertainty.org/). I would like to thank Ozzie Gooen, Nuño Sempere for their feedback on this project. I'd like to thank the rest of the Squiggle development team: Ozzie Gooen, Nuño Sempere and Quinn Dougerty. I'd further like to thank David Reinstein for his feedback and interest on this project.\n\n![Quantified Uncertainty Research Institute](https://super-static-assets.s3.amazonaws.com/09bb1362-5e3f-4724-8ffd-f3235f67356f/images/6151ac3e-aed7-44c7-9827-399fe6e9222b.png)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":688,"value":"## Footnotes","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":709,"value":"footnote = (referencingCellName, number, text) => {\n return md`

^ ${text}

`;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":660,"value":"footnote1 = footnote(\"cell-177\", 1, `I use a bit of a trick here with Squiggle, where I use\n[\\`toPointSet\\`](https://www.squiggle-language.com/docs/Features/Functions#converting-between-distribution-formats) to call [KDE](https://en.wikipedia.org/wiki/Kernel_density_estimation). This makes squiggle try to guess a continuous distribution from the 19 examples I have. This is not perfect, as we actually have more information here, particularly that the amount spent on the program cannot be higher than 100%. This distribution does have a small amount of probability mass above 100%. But this is negligible. I'm not sure what KDE looks like knowing this. If you want to learn more about importing data into squiggle notebooks, see [our tutorial](https://observablehq.com/@hazelfire/squiggle#importing_data).`)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":700,"value":"footnote2 = footnote(\"cell-349\", 2,`\nI use the same technique here to make a distribution. Sadly, there is a [bug](https://github.com/quantified-uncertainty/squiggle/issues/445) in squiggle where it will sometimes interpret distribution samples to come from a continuous distribution rather than a mixed one. This bug however should be fixed fairly soon.`)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":722,"value":"footnote(\"cell-220\", 3, `In squiggle, under allmost all situations, distributions are assumed to be indipendent. This becomes an issue, for instance when you are measing log increase in consumption: \\`log(baseline_consumption + amount_consumed) - log(baseline_consumption) \\`.\n\nHere, the distributions that are being subtracted from each other are not indipendent, because they both rely on the same variable \\`baseline_consumption\\`. If they were, you would get nonsense results such as having probability mass below 0.\n\nHowever, sample sets do not have this property, and allow you to do operations on distributions that are not indipendent. This is why I convert \\`baseline_consumption\\` to a sample set\n\nTo take a look at the different types of distributions available in squiggle, [check out our documentation](https://www.squiggle-language.com/docs/Discussions/Three-Formats-Of-Distributions)`)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":840,"value":"footnote4 = footnote(\"cell-485\", 4, `Another reason that I prefer the continuous format is because I love Dimensional Analysis (checking units of calcualtions), and the continuous version is [much easier to solve units for](https://forum.effectivealtruism.org/posts/WmQwtYEajNDuPdyZx/type-checking-givewell-s-givedirectly-cost-effective#Problems_and_caveats).`)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":753,"value":"footnote5 = footnote(\"cell-503\",5,`I'm not sold that this is an important consideration and could complicate the model. Removal of this consideration would be a change I would make.`)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":551,"value":"## Appendix A, Squiggle features that could improve this model.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":138,"value":"Current rooms for improvement, particularly within Squiggle development:\n - It would be lovely if I could just see the mean of a distribution, so that I could check whether there are any math errors I should know about.\n - Squiggle represents some collections of samples as mixed distributions when they should be entirely discrete\n - I use `toPointSet(fromSamples(x))` to fit a distribution on x, when I shouldn't really need to do that. A `fitFromSamples(x)` or something of the like would be preferable.\n - Allow writing bindings and returns in one cell.\n - Shorten the floats printed in Symbolic representations of distributions.\n - The performance is pretty slow at this point. It would be valuable to improve it.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":934,"value":"$38","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"}],"files":"$W39","stdlib":"1","titling":"auto"}} 33:"$32:body"