viewof all_charities = cell(`givedirectly = ${importSampleSetDist(FileAttachment("givedirectly_effectiveness.csv"))}
new_incentives = ${importSampleSetDist(newIncentivesProject, "units_of_value_generated_per_dollar_spent_by_new_incentives", results_after_leverage_or_funging_adjustment)}
helen_keler_international = ${importSampleSetDist(helenKelerInternationalProject, "average_total_value", average_total_value)}
malaria_consortium = ${importSampleSetDist(malariaConsortiumProject, "global_mean_units_of_value_generated_per_philanthropic_dollar_spent_after_accounting_for_leverage_and_funging", global_mean_units_of_value_generated_per_philanthropic_dollar_spent_after_accounting_for_leverage_and_funging)}
Plot.dist({show: [{name: "GiveDirectly", value: givedirectly}, {name: "New Incentives", value: new_incentives},{name: "Malaria Consortium", value: malaria_consortium}, {name: "Helen Keler International", value: helen_keler_international}]})`)
viewof vpi_givewell = cell(`voi(c) = mean(SampleSet.mapN(c, {|xs| max(xs)}))
vou(c) = max(map(c, {|x| mean(x)}))
vpi(x) = voi(x) - vou(x)
charities = [givedirectly, helen_keler_international, new_incentives]
vpi_givewell = vpi(charities)
vpi_givewell`,all_charities)
viewof cost_ratio = cell(`vpi_cost(x) = {
my_voi = voi(x)
my_voi / (my_voi - vou(x))
}
cost_ratio = vpi_cost(charities)
cost_ratio`, vpi_givewell)
This can be interpreted as "Doing research that completely reduces all uncertainty in GiveWell charities is only worth it if we spend ${formatFloat(run(`cost_ratio`, cost_ratio).value)}x more on grant-making than on the research".
However, this hypothetical research project does not reflect reality, because no research project will ever completely reduce uncertainty. This project also reduces uncertainty in all parameters in all GiveWell models. So this number makes a good upper bound on the amount you should spend on research depending on the problem. How to measure anythinghttps://www.howtomeasureanything.com/ recommends spending 10x less on research than the amount recommended by the value of information.
This number reflects that it's not particularly worthwhile to reduce uncertainty in GiveWell CEAs, and although we can only speculate, we believe GiveWell knows this. This is why GiveWell focuses on investigating new interventions. So below is an example of how to evaluate the value of information on new interventions.
Later we'll do a value of information analysis, which will investigate which parameters are worth researching more. This result finds that all parameters are essentially not worth researching, meaning we don't need to do any more research into GiveWell charities.
viewof fistula_foundation = cell(`fistula_foundation = ${importSampleSetDist(fistulaFoundationProject, "total_value_per_dolar", total_value_per_dollar)}
Plot.dist({show: [{name: "New Incentives", value: new_incentives}, {name: "Malaria Consortium", value: malaria_consortium}, {name: "Helen Keler International", value: helen_keler_international}, {name: "Fistula Foundation", value: fistula_foundation}]})`, all_charities, {showSummary: true})
viewof cost_ratio_fistula = cell(`cost_ratio_fistula = vpi_cost([SampleSet.fromDist(mx(vou([givedirectly, new_incentives, helen_keler_international, fistula_foundation]))), fistula_foundation])`, [fistula_foundation, cost_ratio, vpi_givewell, all_charities])
Therefore, it's worth reducing all uncertainty in the fistula foundation if you have ${formatFloat(run(`cost_ratio_fistula`, cost_ratio_fistula).value)}x more money in grantmaking than the research costs. That much more worthwhile!
However, there are several uncertain variables within the fistula foundation CEA. Which one should we research? We can do a value of information analysis!
fistulaCell(`mapDict(x, f) = Dict.fromList(map(Dict.keys(x), {|key| [key, f(value)]}))
to_mean(x) = mapDict(x, mean)
vpi_analysis(f, params, other) = {
mean_params = to_mean(params)
vou = max([mean(f(params)), other])
Dict.fromList(map(Dict.keys(mean_params), {|key| max(mean(SampleSet.map(params[key], {|s| f(Dict.merge(mean_params, Dict.fromList([[key, s]])))})), other) - vou}))
}
vpi_analysis(total_value_per_dollar, params, vou(charities))`, [params, total_value_per_dollar, all_charities])
importSampleSetDist = (file) => `SampleSet.fromList([${file.csv().map(x => x.dist)}])`
import {toc} from "@harrystevens/toc"
formatFloat = x => new Intl.NumberFormat('en-US', {maximumSignificantDigits: 3}).format(x)
One platform to build and deploy the best data apps
Experiment and prototype by building visualizations in live JavaScript notebooks. Collaborate with your team and decide which concepts to build out.
Use Observable Framework to build data apps locally. Use data loaders to build in any language or library, including Python, SQL, and R.
Seamlessly deploy to Observable. Test before you ship, use automatic deploy-on-commit, and ensure your projects are always up-to-date.