### Adjustment for Self-Reporting Bias in the Randomised Controlled Trial (RCT) of New Incentives' Program
Following are adjustments for self-reporting bias, which [GiveWell expects](https://www.givewell.org/international/technical/programs/new-incentives#Self-reporting_bias) to contribute to a net overestimation of vaccination in the control group, leading to an underestimation of the effect of New Incentives' program on vaccination rates. Vaccination coverage in control areas is a key input for cost analysis so its overestimation can also affect calculation of cost-effectiveness. Overall, self-report bias - as estimated by comparison of carers' reports of children receiving the BCG vaccine to children's rates of scarring from the BCG vaccine - adjusts the increase in vaccination rate from formatFloat2(run(`mean(vaccination_increase_raw * 100)`, vaccination_increase_raw).value) pp to approx. formatFloat2(run(`mean(est_true_effect_based_on_bcg_scarring * 100)`, est_true_effect_based_on_bcg_scarring).value) pp. This estimation is higher than GiveWell's 20pp. This may be because the IDInsight's report presents a 95% CI for each finding as well as a 'value' which is not the average of the upper and lower bounds. This suggests an asymmetric distribution which GiveWell has accounted for by using the main value presented while we have used the entire range, approximating it to a log normal distribution. This could be improved by reviewing the type of distribution used.
Approximately formatFloat2(run(`mean(bcg_scarring_control * 100)`, adjustment_self_report_bias).value)% of infants who receive the BCG vaccine will bear a small scar at the injection site which can be used as evidence of their vaccination. However, sometimes infants may be identified as having a scar when they do not or visa versa and this must also be adjusted for. The proportion of children who scar from the BCG vaccine appears to be higher in the areas treated by the program than in the control areas. The reason for this is uncertain but it may indicate some infants receiving the vaccine more than once.
- refName({adjustment_self_report_bias}, `bcg_scarring_control`): The proportion of BCG *vaccinated* children in the control group who displayed scars
This value has a high level of uncertainty but is based on review of literature from around the world with scarring rates ranging from 80.4% in Pakistan to 98.6% in Peru. GiveWell assumes the rate in North West Nigeria to be comparable to the 90% found in [Dhanawade et al.'s 2015 report](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4535100/) in infants in India, although it may be worth noting that lower birthweight infants in India had scarring rates as low as 45%. Low birthweight can be contributed to by the birth-giver being underweight or underage, or having poor nutrition in pregnancy. Poorer people are at a greater risk of these factors.
- refName({adjustment_self_report_bias}, `bcg_scarring_treatment`): The proportion of BCG *vaccinated* children in the treatment group who displayed scars
This value has been calculated by comparing scars to self-reports, Child Healthcare Cards, and clinic immunisation records. [GiveWell attributes](https://www.givewell.org/international/technical/programs/new-incentives#footnote16_9txuuzy) the ostensibly higher rate of scarring in the treatment group than the control to "some infants in the treatment group receiving BCG more than once, either due to errors (e.g. unclear child health cards leading clinic staff to believe that an infant had not yet received BCG when they in fact had) or repeat enrollment, though this is speculative." Since infants would have a certain probability of scaring each time they received the vaccine, a portion of the children being vaccinated multiple times would increase the proportion of vaccinated children with a scar.
I've set the distribution for this to a beta distribution with a mean of 0.97 and standard deviations of 0.02 so that the 95% confidence interval does not spill over 1, as this would be impossible.
- ${refName({adjustment_self_report_bias}, `scarring_detection_rate`)} (sensitivity): The proportion of BCG scars the researchers expected to be able to detect.
Some infants may have scarred but had their scars either not detected by research staff or mistaken for something else, leading to an underestimation of BCG scarring and therefore of vaccination - if this was calculated based on scarring. This parameter accounts for that source of error.
I've taken the 95% CI for this from [IDInsights report](https://files.givewell.org/files/DWDA%202009/NewIncentives/IDinsight_Impact_Evaluation_of_New_Incentives_Final_Report.pdf). This resulted in a mean lower than the value used by GiveWell.
- ${refName({adjustment_self_report_bias}, `scarring_identification_correct`)} (specificity): The proportion of BCG scars detected that were actually BCG scars, or equivalently, the proportion of infants who did not scar who were correctly detected as 'not scarring'.
Some infants recorded as having scars from the BCG vaccine may not have actually. What the researchers identified as a BCG scar may have actually been a birth mark or a scar from another source. It's also possible researchers could have mistakenly counted the same child with a scar twice. These mistakes would lead to an overestimation of BCG scarring and therefore of vaccination - if this was calculated based on scarring. This parameter accounts for that source of error.
I've taken the 95% CI for this from [IDInsights report](https://files.givewell.org/files/DWDA%202009/NewIncentives/IDinsight_Impact_Evaluation_of_New_Incentives_Final_Report.pdf). This resulted in a mean lower than the value used by GiveWell.
- ${refName({adjustment_self_report_bias}, `bcg_scars_detected_in_control`)}: The proportion of *all* children in the control cohort with BCG scars detected
I've taken the 95% CI for this from [IDInsights report](https://files.givewell.org/files/DWDA%202009/NewIncentives/IDinsight_Impact_Evaluation_of_New_Incentives_Final_Report.pdf). This resulted in a mean higher than the value used by GiveWell.
- ${refName({adjustment_self_report_bias}, `bcg_scars_detected_in_treatment`)}: The proportion of *all* children in the treatments cohort with BCG scars detected
I've taken the 95% CI for this from [IDInsights report](https://files.givewell.org/files/DWDA%202009/NewIncentives/IDinsight_Impact_Evaluation_of_New_Incentives_Final_Report.pdf). This resulted in a mean higher than the value used by GiveWell.
- ${refName({est_bcg_vaccination_in_control})}: The proportion of children in the control group estimated to have received the BCG vaccine (based on scarring)
- ${refName({est_bcg_vaccination_in_treatment})}: The proportion of children in the treatment group estimated to have received the BCG vaccine (based on scarring)
- ${refName({est_true_effect_based_on_bcg_scarring})}: The estimated true effect of the program, i.e. the increase in vaccination as a result of the program, calculated by the differences in vaccination rates (as estimated based on scarring) between the control and treatment groups.