Sensitivity Analysis to Unobserved Confounders: A Comparative Review to Estimate Confounding Strength in Sensitivity Models

This repository contains the R implementation of Sensitivity Analysis to Unobserved Confounders: A Comparative Review to Estimate Confounding Strength in Sensitivity Models. IMPORTANT: For licensing reasons, some parts of the code were changed or removed (figure generation, random forests, quantile regression and parallel computing). Therefore, the present code is NOT FUNCTIONAL. To get a working code, see instructions in src/utils.R.

Note: As informal benchmarking via random forests was removed from this implementation, slight variations with respect to the results from the paper can be experienced.

• Abstract
• Description
• Requirements and Licenses
• Output of the sessionInfo() command

Abstract

Causal inference is only valid when its underlying assumptions are satisfied, one of the most central being the ignorability assumption (also known as unconfoundedness or exogeneity). In practice, however, this assumption is often unrealistic in observational studies, as some confounding variables may remain unobserved. To address this limitation, sensitivity models for Inverse Probability Weighting (IPW) estimators, known as Marginal Sensitivity Models, have been introduced, allowing for a controlled relaxation of ignorability. Over the past decades, a substantial body of literature has emerged around these models, aiming to derive sharp and robust bounds for both binary and continuous treatment effects. A key element of these approaches is the specification of a sensitivity parameter, sometimes referred to as the ``confounding strength", which quantifies the extent of deviation from ignorability. Yet, determining an appropriate value for this parameter is challenging, and the final interpretation of sensitivity analyses can be unclear. We believe these difficulties represent major obstacles to the adoption of such methods in practice. In this review, after introducing sensitivity analyses for IPW estimators, we focus on different strategies to estimate or lower bound the confounding strength, select the most suitable approach, and avoid common pitfalls in the interpretation of results.

Description

This repository is related to the paper Sensitivity Analysis to Unobserved Confounders: A Comparative Review to Estimate Confounding Strength in Sensitivity Models, Baitairian et al. (2025). It is organized as follows:

• confounding_strength_estimation.Rmd: an R Markdown document to reproduce part of the figures and numerical results from the paper. Users need to modify src/utils.R before executing code from this file;
• src/utils.R: functions that must be redefined by the user to get a working implementation;
• src/data_simulation.R: functions to create the simulated datasets;
• src/informal_benchmarking.R: functions to perform informal benchmarking via logistic regression;
• src/quantile_balancing.R: functions used to compute the sensitivity bounds from Dorn and Guo (2023);
• src/negative_control_lb.R: functions to estimate a lower bound on the confounding strength $\Gamma$ via negative control outcomes;
• src/rct_lb.R: functions to estimate a lower bound on the confounding strength $\Gamma$ via Randomized Controlled Trials (RCTs), following De Bartolomeis et al. (2024);
• src/monte_carlo_fun.R: functions to estimate or lower bound the confounding strength with the three methods on several Monte-Carlo samples;
• src/mc_experiments.R: a file to execute the 14 experiments from the paper;
• src/mc_experiments_table.R: a file to get the percentage of inclusion of the null effect and of the ATE in the sensitivity bounds under unconfoundedness and with the true confounding strength;
• src/mc_experiments_table_sa_with_estim.R: a file to get the percentage of inclusion of the null effect and of the ATE in the sensitivity bounds for each estimated $\Gamma$ (median estimation per method);
• /images: a folder to save the generated plots;
• /images/article_figures/: a folder containing the original plots from the article;
• /data: a folder that contains the Monte-Carlo data generated for each experiment;
• /results: a folder that contains the results of the sensitivity analyses.

Note: the bounds from Dorn and Guo (2023) and the method from De Bartolomeis et al. (2024) were reimplemented.

References

Dorn, J., & Guo, K. (2023). Sharp sensitivity analysis for inverse propensity weighting via quantile balancing. Journal of the American Statistical Association, 118(544), 2645-2657. Link

De Bartolomeis, P., Martinez, J. A., Donhauser, K., & Yang, F. (2024, April). Hidden yet quantifiable: A lower bound for confounding strength using randomized trials. In International Conference on Artificial Intelligence and Statistics (pp. 1045-1053). PMLR. Link

Requirements and licenses

This repository is under a non-commercial license (see LICENSE.txt file). The following libraries are used in this repository with R version 4.3.2.

Library	Version	License
foreach	1.5.2	Apache License 2.0
ggplot2	3.4.4	MIT
latex2exp	0.9.6	MIT

Output of the sessionInfo() command

R version 4.3.2 (2023-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Rocky Linux 8.7 (Green Obsidian)

Matrix products: default
BLAS/LAPACK: /usr/lib64/libopenblas-r0.3.15.so;  LAPACK version 3.9.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8        LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8    LC_PAPER=C.UTF-8      
 [8] LC_NAME=C              LC_ADDRESS=C           LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

time zone: UTC
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] foreach_1.5.2   latex2exp_0.9.6 ggplot2_3.4.4  

loaded via a namespace (and not attached):
 [1] vctrs_0.6.5       cli_3.6.2         knitr_1.45        rlang_1.1.2       xfun_0.41         stringi_1.8.3     generics_0.1.3    doSNOW_1.0.20     glue_1.6.2       
[10] colorspace_2.1-0  htmltools_0.5.7   snow_0.4-4        fansi_1.0.6       scales_1.3.0      rmarkdown_2.25    grid_4.3.2        evaluate_0.23     munsell_0.5.0    
[19] tibble_3.2.1      fastmap_1.1.1     yaml_2.3.8        lifecycle_1.0.4   stringr_1.5.1     compiler_4.3.2    dplyr_1.1.4       codetools_0.2-19  pkgconfig_2.0.3  
[28] rstudioapi_0.15.0 digest_0.6.34     R6_2.5.1          tidyselect_1.2.0  utf8_1.2.4        pillar_1.9.0      parallel_4.3.2    magrittr_2.0.3    withr_2.5.2      
[37] tools_4.3.2       gtable_0.3.4      iterators_1.0.14