research.qmd

---
title: "Research Examples"
---

```{=html}
<style>
.paper-grid {
  display: grid;
  grid-template-columns: repeat(2, 1fr);
  gap: 1rem;
  margin-bottom: 1.5rem;
}
@media (max-width: 768px) {
  .paper-grid { grid-template-columns: 1fr; }
}
.paper-card {
  border: 1px solid #dee2e6;
  border-left: 4px solid #2780e3;
  border-radius: 6px;
  padding: 1.1rem 1.4rem;
  background: #fff;
  transition: box-shadow 0.15s;
  display: flex;
  flex-direction: column;
}
.paper-card:hover {
  box-shadow: 0 2px 12px rgba(0,0,0,0.08);
}
.paper-type {
  font-size: 0.72rem;
  font-weight: 600;
  text-transform: uppercase;
  letter-spacing: 0.06em;
  color: #6c757d;
  margin-bottom: 0.5rem;
}
.paper-title {
  font-size: 1rem;
  font-weight: 600;
  margin-bottom: 0.3rem;
  line-height: 1.4;
}
.paper-title a {
  color: #212529;
  text-decoration: none;
}
.paper-title a:hover {
  color: #2780e3;
}
.paper-meta {
  font-size: 0.82rem;
  color: #6c757d;
  margin-bottom: 0.2rem;
}
.paper-journal {
  font-size: 0.85rem;
  font-style: italic;
  color: #495057;
  margin-bottom: 0.75rem;
  flex: 1;
}
.paper-abstract {
  display: none;
  font-size: 0.82rem;
  color: #495057;
  line-height: 1.55;
  margin-bottom: 0.75rem;
  padding: 0.6rem 0.75rem;
  background: #f8f9fa;
  border-radius: 4px;
  border-left: 3px solid #dee2e6;
}
.paper-links {
  display: flex;
  gap: 0.5rem;
  flex-wrap: wrap;
  margin-top: auto;
}
.paper-btn {
  display: inline-block;
  font-size: 0.78rem;
  font-weight: 500;
  padding: 0.25rem 0.7rem;
  border-radius: 4px;
  text-decoration: none;
  border: 1px solid;
  transition: background-color 0.15s, color 0.15s;
  white-space: nowrap;
}
.paper-btn-journal {
  border-color: #2780e3;
  color: #2780e3;
  background: transparent;
}
.paper-btn-journal:hover {
  background: #2780e3;
  color: #fff;
}
.paper-btn-preprint {
  border-color: #6c757d;
  color: #6c757d;
  background: transparent;
}
.paper-btn-preprint:hover {
  background: #6c757d;
  color: #fff;
}
.paper-btn-abstract {
  border-color: #adb5bd;
  color: #6c757d;
  background: transparent;
  cursor: pointer;
  font-family: inherit;
}
.paper-btn-abstract:hover, .paper-btn-abstract.active {
  background: #e9ecef;
  color: #212529;
  border-color: #6c757d;
}
.badge-oa {
  display: inline-block;
  font-size: 0.7rem;
  font-weight: 600;
  font-style: normal;
  padding: 0.15rem 0.5rem;
  border-radius: 20px;
  background: #d4edda;
  color: #155724;
  border: 1px solid #b8dac3;
  vertical-align: middle;
  margin-left: 0.4rem;
  letter-spacing: 0.02em;
}
</style>
<script>
function toggleAbstract(id, btn) {
  var el = document.getElementById(id);
  var shown = el.style.display === 'block';
  el.style.display = shown ? 'none' : 'block';
  btn.classList.toggle('active', !shown);
  btn.textContent = shown ? 'Abstract' : 'Abstract ▲';
}
</script>
```

## Key papers

<div class="paper-grid">

<div class="paper-card">
<div class="paper-type">Core IRW resource</div>
<div class="paper-title"><a href="https://link.springer.com/article/10.3758/s13428-025-02796-y" target="_blank">An introduction to the Item Response Warehouse (IRW): A resource for enhancing data usage in psychometrics</a></div>
<div class="paper-meta">Domingue B, Braginsky M, Caffrey-Maffei L, Gilbert JB, Kanopka K, Kapoor R, Lee H, Liu Y, Nadela S, Pan G, Zhang L, Zhang S, Frank MC · 2025</div>
<div class="paper-journal">Behavior Research Methods <span class="badge-oa">Open Access</span></div>
<div class="paper-abstract" id="abstract-1">The Item Response Warehouse (IRW) is a collection and standardization of a large volume of item response datasets in a free and open-source platform for researchers. We describe key elements of the data standardization process and provide a brief description of the over 900 datasets in the current iteration of the IRW (version 28.2). We describe how to access the data through both the website and an API, and offer a brief tutorial with example R code illustrating how to download data from the IRW and use it in standard psychometric analyses. While we are continuing to develop the IRW, this presentation may help researchers utilize data from this resource for work in psychometrics and related fields.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-1', this)">Abstract</button>
<a href="https://link.springer.com/article/10.3758/s13428-025-02796-y" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-type">Additional resources & item text</div>
<div class="paper-title"><a href="https://www.ce-jeme.org/journal/vol7/iss1/3" target="_blank">The Item Response Warehouse: What It Is, How to Use It, and Targets for Potential Improvements</a></div>
<div class="paper-meta">Nadela S, Lee H, Jain N, Gupta A, Zhang X, Domingue B · 2026</div>
<div class="paper-journal">Chinese/English Journal of Educational Measurement and Evaluation <span class="badge-oa">Open Access</span></div>
<div class="paper-abstract" id="abstract-2">The Item Response Warehouse (IRW) is a repository of harmonized item response datasets designed to support secondary analysis and methodological research in psychological and educational measurement. This paper serves as a practical guide for researchers interested in using the IRW. We describe the structure of IRW datasets and the quantitative and qualitative metadata available for dataset selection, and we demonstrate how researchers can navigate the IRW website to explore and compare available tables. We further show how the IRW R and Python packages can be used to filter datasets programmatically, download response-level data, and generate standardized citations for reproducible research workflows. Finally, we describe additional IRW features, including access to item text and ongoing development efforts, that are designed to both support immediate use of the IRW and allow for community input related to its continued expansion.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-2', this)">Abstract</button>
<a href="https://www.ce-jeme.org/journal/vol7/iss1/3" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
</div>
</div>

</div>

## Psychometrics research using IRW data

<div class="paper-grid">

<div class="paper-card">
<div class="paper-title"><a href="https://osf.io/preprints/psyarxiv/r9b3y_v1" target="_blank">Comparing Compensatory and Non-Compensatory MIRT Models at Scale: Evidence from IRW</a></div>
<div class="paper-meta">Liu Y, Zhang L, Domingue B · 2026</div>
<div class="paper-journal">PsyArXiv</div>
<div class="paper-abstract" id="abstract-3">A central choice in multidimensional item response theory (MIRT) concerns how multiple latent skills combine to produce success: compensatory models allow trade-offs across dimensions, whereas fully noncompensatory models impose a conjunctive constraint in which limited proficiency on any required dimension can constrain success. Despite longstanding conceptual interest, there is limited evidence on how these skill-combination assumptions affect out-of-sample predictive performance across large-scale assessments. Using the Item Response Warehouse (IRW) and a common missing-responses cross-validation framework, we compare a unidimensional 2PL baseline with compensatory and fully noncompensatory two-dimensional 2PL specifications and summarize predictive differences with the InterModel Vigorish (IMV). Across datasets, the compensatory specification yields small but consistently positive predictive improvements over the unidimensional baseline, whereas the fully noncompensatory specification rarely improves prediction. Item-level analyses further reveal within-test heterogeneity, suggesting that conjunctive structure tends to be localized to subsets of items rather than test-wide.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-3', this)">Abstract</button>
<a href="https://osf.io/preprints/psyarxiv/r9b3y_v1" class="paper-btn paper-btn-preprint" target="_blank">Preprint →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-title"><a href="https://journals.sagepub.com/doi/10.1177/00131644261440556" target="_blank">Signposts on the Path from Nominal to Ordinal Scales</a></div>
<div class="paper-meta">Nalbandyan R, Gilbert JB, Franco VR, Domingue BW · 2026</div>
<div class="paper-journal">Educational and Psychological Measurement</div>
<div class="paper-abstract" id="abstract-4">Polytomous item response data are typically classified as either nominal or ordinal, but this binary distinction may oversimplify their true structure. In this paper, we reframe the nominal–ordinal distinction as a continuum and introduce six empirical indices to quantify the degree of category ordering in item response data. Through extensive simulations with various IRT models and applications to 245 empirical datasets, we evaluate the indices' sensitivity, computational efficiency, and interpretability across diverse measurement contexts. Our findings show that two parametric indices are particularly robust and informative, even with low-frequency categories. These indices offer a practical tool for assessing whether and how item categories align with ordinal assumptions, supporting more accurate measurement and model selection. We conclude that treating ordering as a continuum, rather than a binary property, provides deeper insights for psychometric practice.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-4', this)">Abstract</button>
<a href="https://journals.sagepub.com/doi/10.1177/00131644261440556" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
<a href="https://osf.io/preprints/psyarxiv/zbv8f" class="paper-btn paper-btn-preprint" target="_blank">Preprint →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-title"><a href="https://onlinelibrary.wiley.com/doi/10.1002/pam.70025" target="_blank">Estimating Heterogeneous Treatment Effects with Item-Level Outcome Data: Insights from Item Response Theory</a></div>
<div class="paper-meta">Gilbert J, Himmelsbach Z, Soland J, Joshi M, Domingue B · 2025</div>
<div class="paper-journal">Journal of Policy Analysis and Management</div>
<div class="paper-abstract" id="abstract-5">Analyses of heterogeneous treatment effects (HTE) are common in applied causal inference research. However, when outcomes are latent variables assessed via psychometric instruments such as educational tests, standard methods ignore the potential HTE that may exist among the individual items of the outcome measure. Failing to account for "item-level" HTE can lead to both underestimated standard errors and identification challenges in the estimation of treatment-by-covariate interaction effects. We demonstrate how Item Response Theory (IRT) models that estimate a treatment effect for each assessment item can both address these challenges and provide new insights into HTE generally. This study uses 75 datasets from 48 randomized controlled trials containing 5.8 million item responses in economics, education, and health research. Our results show that the item-level HTE model reveals item-level variation masked by single-number scores, provides more meaningful standard errors, allows for estimates of the generalizability of causal effects to untested items, and provides estimates of standardized treatment effect sizes corrected for attenuation due to measurement error.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-5', this)">Abstract</button>
<a href="https://onlinelibrary.wiley.com/doi/10.1002/pam.70025" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
<a href="https://arxiv.org/abs/2405.00161" class="paper-btn paper-btn-preprint" target="_blank">Preprint →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-title"><a href="https://psycnet.apa.org/doiLanding?doi=10.1037%2Fmet0000764" target="_blank">Estimating Causal Effects on Psychological Networks Using Item Response Theory</a></div>
<div class="paper-meta">Gilbert J, Domingue B, Kim J · 2025</div>
<div class="paper-journal">Psychological Methods</div>
<div class="paper-abstract" id="abstract-6">Network models in which each variable interacts with the others in a complex system have emerged as an important alternative to latent variable models in psychometric research. However, confirmatory methods for group network comparison can be limited by practical constraints, such as the computational intractability of the Ising model in large networks. In this study, we demonstrate how to estimate causal effects on network state and strength when direct network estimation is not feasible by leveraging the mathematical equivalencies between the Ising model and item response theory (IRT) models. We demonstrate through simulation that a two-parameter logistic (2PL) explanatory IRT model can simultaneously recover causal effects on network state and strength. We then replicate our approach with 72 empirical datasets from randomized controlled trials in education, economics, health, and related fields. Our results show that causal effects on network strength are both common and uncorrelated with effects on network state, suggesting that causal network models can provide new insight into the impact of interventions in the social and behavioral sciences.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-6', this)">Abstract</button>
<a href="https://psycnet.apa.org/doiLanding?doi=10.1037%2Fmet0000764" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
<a href="https://osf.io/preprints/psyarxiv/7k6xz" class="paper-btn paper-btn-preprint" target="_blank">Preprint →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-title"><a href="https://meth.psychopen.eu/index.php/meth/article/view/15773" target="_blank">How Measurement Affects Causal Inference: Attenuation Bias Is (Usually) More Important Than Outcome Scoring Weights</a></div>
<div class="paper-meta">Gilbert J · 2025</div>
<div class="paper-journal">Methodology <span class="badge-oa">Open Access</span></div>
<div class="paper-abstract" id="abstract-7">When analyzing treatment effects on outcome variables constructed from psychometric instruments (e.g., educational test scores, psychological surveys, or patient reported outcomes), researchers face many choices and competing guidance for scoring the measures and modeling results. This study examines the impact of outcome measure scoring and modeling approaches through simulation and an empirical application. Results show that estimates from multiple methods applied to the same data will vary because two-step models using sum or factor scores provide attenuated standardized treatment effects compared to latent variable models. This bias dominates any other differences between models or features of the data generating process, such as the use of scoring weights. An errors-in-variables correction removes the bias from two-step models. An empirical application to 10 datasets from randomized controlled trials demonstrates the sensitivity of the results to model selection. This study shows that the psychometric principles most consequential in causal inference are related to attenuation bias rather than optimal outcome scoring weights.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-7', this)">Abstract</button>
<a href="https://meth.psychopen.eu/index.php/meth/article/view/15773" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-title"><a href="https://journals.sagepub.com/doi/10.1177/00131644261426972" target="_blank">Conditional Dependencies Between Response Time and Item Discrimination: An Item-Level Meta-Analysis</a></div>
<div class="paper-meta">Gilbert J, Young W, Himmelsbach Z, Ulitzsch E, Domingue B · 2025</div>
<div class="paper-journal">Educational and Psychological Measurement</div>
<div class="paper-abstract" id="abstract-8">The use of process data such as response time (RT) in psychometrics has generally focused on the relationship between speed and accuracy. The potential relationships between RT and item discrimination remain less explored. In this study, we propose a model for simultaneously estimating the relationships between RT and item discrimination at the person, item, and person-by-item (residual) levels and illustrate our approach through an item-level meta-analysis of 40 empirical datasets comprising 1.84 million item responses. We find no evidence of average differences in item discrimination between items of different time intensity or persons of different average RT, while residual RT strongly and negatively predicts item discrimination. While heterogeneity is high, we find little evidence of moderation by overall dataset characteristics. Our results suggest that RT data can provide insights into the measurement properties of educational and psychological assessments, but that the relationships between RT and item discrimination are highly variable.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-8', this)">Abstract</button>
<a href="https://journals.sagepub.com/doi/10.1177/00131644261426972" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
<a href="https://osf.io/preprints/psyarxiv/rp34w_v1" class="paper-btn paper-btn-preprint" target="_blank">Preprint →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-title"><a href="https://journals.sagepub.com/doi/abs/10.3102/10769986251393339" target="_blank">Item-Level Heterogeneity in Value Added Models: Implications for Reliability, Cross-Study Comparability, and Effect Sizes</a></div>
<div class="paper-meta">Gilbert J, Himmelsbach Z, Miratrix L, Ho AD, Domingue B · 2025</div>
<div class="paper-journal">Journal of Educational and Behavioral Statistics</div>
<div class="paper-abstract" id="abstract-9">Value added models (VAMs) attempt to estimate the causal effects of teachers and schools on student test scores. We apply Generalizability Theory to show how estimated VA effects depend upon the selection of test items. Standard VAMs estimate causal effects on the items that are included on the test. Generalizability demands consideration of how estimates would differ had the test included alternative items. We use item-level data from the IRW to estimate the item-level heterogeneity in VA effects and explore implications for reliability, cross-study comparability, and effect sizes.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-9', this)">Abstract</button>
<a href="https://journals.sagepub.com/doi/abs/10.3102/10769986251393339" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
<a href="https://doi.org/10.26300/ez4q-fs31" class="paper-btn paper-btn-preprint" target="_blank">Preprint →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-title"><a href="https://onlinelibrary.wiley.com/doi/10.1111/emip.70011" target="_blank">The Sensitivity of Value-Added Estimates to Test Scoring Decisions</a></div>
<div class="paper-meta">Gilbert J, Soland J, Domingue B · 2025</div>
<div class="paper-journal">Educational Measurement: Issues and Practice</div>
<div class="paper-abstract" id="abstract-10">Value-Added Models (VAMs) are both common and controversial in education policy and accountability research. While the sensitivity of VAMs to model specification and covariate selection is well documented, the extent to which test scoring methods (e.g., mean scores vs. IRT-based scores) may affect VA estimates is less studied. We examine the sensitivity of VA estimates to scoring method using empirical item response data from 18 education datasets. We show that VA estimates are frequently highly sensitive to scoring method, holding constant students and items.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-10', this)">Abstract</button>
<a href="https://onlinelibrary.wiley.com/doi/10.1111/emip.70011" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
<a href="https://doi.org/10.26300/g4gn-s810" class="paper-btn paper-btn-preprint" target="_blank">Preprint →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-title"><a href="https://link.springer.com/article/10.1007/s11336-024-09977-2" target="_blank">The InterModel Vigorish as a lens for understanding (and quantifying) the value of item response models for dichotomously coded items</a></div>
<div class="paper-meta">Domingue B, Kanopka K, Kapoor R, Pohl S, Chalmers P, Rahal C, Rhemtulla M · 2024</div>
<div class="paper-journal">Psychometrika</div>
<div class="paper-abstract" id="abstract-11">The deployment of statistical models, such as those used in item response theory (IRT), necessitates the use of indices that are informative about the degree to which a given model is appropriate for a specific data context. We introduce the InterModel Vigorish (IMV) as an index that can be used to quantify accuracy for models of dichotomous item responses based on the improvement across two sets of predictions. This index has a range of desirable features: it can be used for the comparison of non-nested models and its values are highly portable and generalizable. We use this fact to compare predictive performance across a variety of simulated data contexts and also demonstrate qualitative differences in behavior between the IMV and other common indices (e.g., the AIC and RMSEA). We also illustrate the utility of the IMV in empirical applications with data from 89 dichotomous item response datasets, helping illustrate how the IMV can be used in practice and substantiating claims regarding various aspects of model performance.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-11', this)">Abstract</button>
<a href="https://link.springer.com/article/10.1007/s11336-024-09977-2" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
<a href="https://osf.io/preprints/psyarxiv/q3djt" class="paper-btn paper-btn-preprint" target="_blank">Preprint →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-title"><a href="https://www.tandfonline.com/doi/full/10.1080/19345747.2024.2361337" target="_blank">Heterogeneity of item-treatment interactions masks complexity and generalizability in randomized controlled trials</a></div>
<div class="paper-meta">Ahmed I, Bertling M, Zhang L, Ho A, Loyalka P, Xue H, Rozelle S, Domingue B · 2024</div>
<div class="paper-journal">Journal of Research on Educational Effectiveness</div>
<div class="paper-abstract" id="abstract-12">Researchers use test outcomes to evaluate the effectiveness of education interventions across numerous randomized controlled trials (RCTs). Aggregate test data—for example, simple measures like the sum of correct responses—are compared across treatment and control groups to determine whether an intervention has had a positive impact on student achievement. We show that item-level data and psychometric analyses can provide information about treatment heterogeneity and improve design of future experiments. We demonstrate heterogeneity of item-treatment interactions in empirical data and discuss implications for the complexity and generalizability of RCT findings.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-12', this)">Abstract</button>
<a href="https://www.tandfonline.com/doi/full/10.1080/19345747.2024.2361337" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
<a href="https://edworkingpapers.com/ai23-754" class="paper-btn paper-btn-preprint" target="_blank">Preprint →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-title"><a href="https://psycnet.apa.org/record/2028-00925-001" target="_blank">A comparison of the predictive performance of continuous and class-based latent trait models</a></div>
<div class="paper-meta">Ma WA, Liu Y, Kanopka K, Ma W, Domingue B · 2025</div>
<div class="paper-journal">Psychological Methods</div>
<div class="paper-abstract" id="abstract-13">The ability of a student can be conceptualized as either a continuously varying entity (e.g., conventional IRT models) or a bundle of latent classes (e.g., cognitive diagnostic models, CDMs). This paper examines the degree to which these approaches—which utilize quite distinctive notions regarding the nature of ability—produce different predictions of response behavior. We first present simulation studies in which CDM-based predictions uniformly outperform those of IRT models when data are generated from CDMs. We then compare CDM- and IRT-based approaches across nine empirical datasets previously analyzed using CDMs. Our findings indicate that overfitting is a pervasive issue across CDM-based predictions, and only a minority of datasets show improved model fit for CDMs over the 2PL model. Researchers and practitioners may need to balance the diagnostic appeal of CDMs with the fact that their complexity can come at the cost of predictive accuracy.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-13', this)">Abstract</button>
<a href="https://psycnet.apa.org/record/2028-00925-001" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
<a href="https://osf.io/preprints/psyarxiv/acwdz_v1" class="paper-btn paper-btn-preprint" target="_blank">Preprint →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-title"><a href="https://osf.io/preprints/psyarxiv/jbhxy_v1" target="_blank">Realistic Simulation of Item Difficulties</a></div>
<div class="paper-meta">Zhang L, Liu Y, Molenaar D, Domingue B · 2025</div>
<div class="paper-journal">PsyArXiv</div>
<div class="paper-abstract" id="abstract-14">Simulation studies are commonly used to improve understanding of psychometric models. For many common models, an essential feature of the simulation is the relative variation in item difficulties. A common practice has been to generate both item difficulty parameters and person parameters directly from a standard normal distribution—an assumption that warrants careful examination. In this paper, leveraging 73 datasets from the Item Response Warehouse, we examine the variability of item difficulty distributions in real-world datasets and investigate how this variation influences estimation and simulation. We identify key distributional characteristics (e.g., variance and skewness) and propose a new method for simulating realistic item difficulties based on empirical data. This method enhances the realism and applicability of simulation results, making them more reflective of real-world measurement conditions and improving the robustness of psychometric model evaluation.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-14', this)">Abstract</button>
<a href="https://osf.io/preprints/psyarxiv/jbhxy_v1" class="paper-btn paper-btn-preprint" target="_blank">Preprint →</a>
</div>
</div>

<div class="paper-card">
<div class="paper-title"><a href="https://journals.sagepub.com/doi/full/10.3102/10769986221099906" target="_blank">Speed–Accuracy Trade-Off? Not So Fast: Marginal Changes in Speed Have Inconsistent Relationships With Accuracy in Real-World Settings</a></div>
<div class="paper-meta">Domingue B, Kanopka K, Stenhaug B, Sulik MJ, Beverly T, Brinkhuis M, Circi R, Faul J, Liao D, McCandliss B, Obradović J, Piech C, Porter T, Soland J, Weeks J, Wise S, Yeatman J · 2022</div>
<div class="paper-journal">Journal of Educational and Behavioral Statistics</div>
<div class="paper-abstract" id="abstract-15">The speed-accuracy tradeoff suggests that responses generated under time constraints will be less accurate. While it has undergone extensive experimental verification, it is less clear whether it applies in settings where time pressures are not being experimentally manipulated. Using a large corpus of 29 response time datasets containing data from cognitive tasks without experimental manipulation of time pressure, we probe whether the speed-accuracy tradeoff holds across a variety of tasks using idiosyncratic within-person variation in speed. We find inconsistent relationships between marginal increases in time spent responding and accuracy; in many cases, marginal increases in time do not predict increases in accuracy. However, we do observe time pressures (in the form of time limits) to consistently reduce accuracy and for rapid responses to typically show the anticipated relationship. We find substantial variation in the item-level associations between speed and accuracy, and on the person side, respondents who exhibit more within-person variation in response speed are typically of lower ability. Collectively, our findings suggest the speed-accuracy tradeoff may be limited as a conceptual model in non-experimental settings.</div>
<div class="paper-links">
<button class="paper-btn paper-btn-abstract" onclick="toggleAbstract('abstract-15', this)">Abstract</button>
<a href="https://journals.sagepub.com/doi/full/10.3102/10769986221099906" class="paper-btn paper-btn-journal" target="_blank">Journal →</a>
<a href="https://osf.io/preprints/psyarxiv/kduv5_v1" class="paper-btn paper-btn-preprint" target="_blank">Preprint →</a>
</div>
</div>

</div>