| Title: | Bayesian Design of Replication Studies |
|---|---|
| Description: | Provides functionality for determining the sample size of replication studies using Bayesian design approaches in the normal-normal hierarchical model (Pawel et al., 2023) <doi:10.1037/met0000604>. |
| Authors: | Samuel Pawel [aut, cre]
|
| Maintainer: | Samuel Pawel <[email protected]> |
| License: | GPL-3 |
| Version: | 0.42.2 |
| Built: | 2025-01-29 04:38:48 UTC |
| Source: | https://github.com/samch93/bayesrepdesign |
"designPrior"
Density method for class "designPrior"
## S3 method for class 'designPrior' density(x, ...)## S3 method for class 'designPrior' density(x, ...)
x |
Object of class |
... |
Other arguments passed to |
Returns the density function of the design prior
Samuel Pawel
dp <- designPrior(to = 2.3123, so = 0.1, mu = 1.1, tau = 0.2) f <- density(dp) tseq <- seq(1, 3.5, 0.01) plot(tseq, f(theta = tseq), type = "l", xlab = "theta", ylab = "Design prior density")dp <- designPrior(to = 2.3123, so = 0.1, mu = 1.1, tau = 0.2) f <- density(dp) tseq <- seq(1, 3.5, 0.01) plot(tseq, f(theta = tseq), type = "l", xlab = "theta", ylab = "Design prior density")
Creates a design prior for the effect size which can then be used for power and sample size calculations of a replication study. The design prior is obtained from updating an initial prior for the effect size by the data from the original study. A normal-normal hierarchical model is assumed, see Pawel et al. (2022) for details.
designPrior( to, so, mu = 0, sp = Inf, tau = 0, g = sp^2/(tau^2 + so^2), h = tau^2/so^2, type = c(NA, "conditional", "predictive", "EB") )designPrior( to, so, mu = 0, sp = Inf, tau = 0, g = sp^2/(tau^2 + so^2), h = tau^2/so^2, type = c(NA, "conditional", "predictive", "EB") )
to |
Effect estimate from original study |
so |
Standard error of effect estimate from original study |
mu |
The initial prior mean. Defaults to |
sp |
The initial prior standard deviation. Defaults to |
tau |
The initial prior heterogeneity standard deviation. Defaults to
|
g |
The relative initial prior variance |
h |
The relative initial prior heterogeneity variance |
type |
Shortcut for special parameter combinations. The available
options are |
The "conditional" design prior corresponds to a point mass at
the original effect estimate, i.e., assuming that the true effect size is
equal to the original effect estimate. The "predictive" design
prior is obtained from updating a uniform initial prior by the likelihood
of the original data. The "EB" design prior is obtained by
empirical Bayes estimation of the variance of the normal prior and
induces adaptive shrinkage that depends on the p-value of the original
effect estimate.
Returns an object of class "designPrior" which is a list containing:
dpMean |
The computed mean of the design prior |
dpVar |
The computed variance of the design prior |
to |
The specified original effect estimate |
so |
The specified original standard error |
mu |
The specified mean of the initial prior |
sp |
The specified standard deviation of the initial prior |
tau |
The specified heterogeneity variance |
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
designPrior(to = 1.1, so = 1)designPrior(to = 1.1, so = 1)
"designPrior"
Plot method for class "designPrior"
## S3 method for class 'designPrior' plot(x, ...)## S3 method for class 'designPrior' plot(x, ...)
x |
Object of class |
... |
Other arguments passed to |
Plots the density of the design prior
Samuel Pawel
dp <- designPrior(to = 2.3123, so = 0.1, mu = 1.1, tau = 0.2) plot(dp) plot(dp, xlim = c(0, 5), length.out = 500)dp <- designPrior(to = 2.3123, so = 0.1, mu = 1.1, tau = 0.2) plot(dp) plot(dp, xlim = c(0, 5), length.out = 500)
This function computes the probabiliy of replication success based on a success region for the replication effect estimate, a design prior, and a replication standard error. If the specified number of sites is larger than 1, the supplied success region has to be formulated in terms of the meta-analytic replication effect estimate across sites.
pors(sregion, dprior, sr, nsites = 1)pors(sregion, dprior, sr, nsites = 1)
sregion |
Success region for replication effect estimate |
dprior |
Design prior object |
sr |
Standard error of replication effect estimate |
nsites |
Number of sites, defaults to |
The probability of replication success
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
dprior <- designPrior(to = 1.1, so = 1) sregion <- successRegion(intervals = cbind(1.96, Inf)) pors(sregion = sregion, dprior = dprior, sr = 1)dprior <- designPrior(to = 1.1, so = 1) sregion <- successRegion(intervals = cbind(1.96, Inf)) pors(sregion = sregion, dprior = dprior, sr = 1)
This function computes the probability to achieve replication success based on a Bayes factor. The Bayes factor is oriented so that values above one indicate evidence for the null hypothesis of the effect size being zero, whereas values below one indicate evidence for the hypothesis of the effect size being non-zero (with normal prior assigned to it).
porsBF01(level, dprior, sr, priormean = 0, priorvar = 1)porsBF01(level, dprior, sr, priormean = 0, priorvar = 1)
level |
Bayes factor level below which replication success is achieved |
dprior |
Design prior object |
sr |
Replication standard error |
priormean |
Mean of the normal prior under the alternative. Defaults to
|
priorvar |
Variance of the normal prior under the alternative. Defaults
to |
The probability to achieve replication success
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552
## specify design prior to1 <- 2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.03) porsBF01(level = 1/10, dprior = dprior, sr = c(0.05, 0.04))## specify design prior to1 <- 2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.03) porsBF01(level = 1/10, dprior = dprior, sr = c(0.05, 0.04))
This function computes the probability to achieve replication success based on the replication Bayes factor. The replication Bayes factor is assumed to be oriented so that values below one indicate replication success, whereas values above one indicate evidence for the null hypothesis.
porsBFr(level, dprior, sr, paradox = TRUE)porsBFr(level, dprior, sr, paradox = TRUE)
level |
Bayes factor level below which replication success is achieved |
dprior |
Design prior object |
sr |
Replication standard error |
paradox |
Should the probability of replication success be computed
allowing for the replication paradox (replication success when the effect
estimates from original and replication study have a different sign)?
Defaults to |
The probability to achieve replication success
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
Verhagen, J. and Wagenmakers, E. J. (2014). Bayesian tests to quantify the result of a replication attempt. Journal of Experimental Psychology: General, 145:1457-1475. doi:10.1037/a0036731
Ly, A., Etz, A., Marsman, M., & Wagenmakers, E.-J. (2018). Replication Bayes factors from evidence updating. Behavior Research Methods, 51(6), 2498-2508. doi:10.3758/s13428-018-1092-x
## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.03) porsBFr(level = 1/10, dprior = dprior, sr = c(0.05, 0.04))## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.03) porsBFr(level = 1/10, dprior = dprior, sr = c(0.05, 0.04))
This function computes the probability to achieve replication success based on the sceptical Bayes factor. The sceptical Bayes factor is assumed to be oriented so that values below one indicate replication success.
porsBFs(level, dprior, sr, paradox = TRUE)porsBFs(level, dprior, sr, paradox = TRUE)
level |
Threshold for the sceptical Bayes factor below which replication success is achieved |
dprior |
Design prior object |
sr |
Replication standard error |
paradox |
Should the probability of replication success be computed
allowing for the replication paradox (replication success when the effect
estimates from original and replication study have a different sign)?
Defaults to |
The probability to achieve replication success
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552
Pawel, S. and Held, L. (2020). The sceptical Bayes factor for the assessement of replication success. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 84(3), 879-911. doi:10.1111/rssb.12491
## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1) porsBFs(level = 1/3, dprior = dprior, sr = 0.05)## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1) porsBFs(level = 1/3, dprior = dprior, sr = 0.05)
This function computes the probability to achieve replication success on equivalence of original and replication effect size. Effect size equivalence is defined by the confidence interval for the difference between the original and replication effect sizes falling within an equivalence region around zero defined by the specified margin.
porsEqu(level, dprior, margin, sr)porsEqu(level, dprior, margin, sr)
level |
1 - confidence level of confidence interval for effect size difference |
dprior |
Design prior object |
margin |
The equivalence margin > 0 for the symmetric equivalence region around zero |
sr |
Replication standard error |
The probability to achieve replication success
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552
Anderson, S. F. and Maxwell, S. E. (2016). There's more than one way to conduct a replication study: Beyond statistical significance. Psychological Methods, 21(1), 1-12. doi:10.1037/met0000051
## specify design prior to1 <- 2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.1) porsEqu(level = 0.1, dprior = dprior, margin = 0.3, sr = c(0.05, 0.03))## specify design prior to1 <- 2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.1) porsEqu(level = 0.1, dprior = dprior, margin = 0.3, sr = c(0.05, 0.03))
This function computes the probability to achieve replication success on statistical significance of the fixed-effect meta-analytic effect estimate obtained from combining original and replication effect estimates.
porsMeta(level, dprior, sr)porsMeta(level, dprior, sr)
level |
Significance level for p-value of the meta-analytic effect estimate (one-sided and in the same direction as the original effect estimate) |
dprior |
Design prior object |
sr |
Replication standard error |
The probability to achieve replication success
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552
## specify design prior to1 <- 2 so1 <- 1 dprior <- designPrior(to = to1, so = so1, tau = 0.1) porsMeta(level = 0.025^2, dprior = dprior, sr = c(0.2, 0.1))## specify design prior to1 <- 2 so1 <- 1 dprior <- designPrior(to = to1, so = so1, tau = 0.1) porsMeta(level = 0.025^2, dprior = dprior, sr = c(0.2, 0.1))
This function computes the probability to achieve replication success based on the sceptical p-value.
porsPs(level, dprior, sr)porsPs(level, dprior, sr)
level |
Threshold for the (one-sided) sceptical p-value below which replication success is achieved |
dprior |
Design prior object |
sr |
Replication standard error |
The sceptical p-value is assumed to be uncalibrated as in Held (2020). The package ReplicationSuccess allows for sample size and power calculations with the recalibrated sceptical p-value (https://CRAN.R-project.org/package=ReplicationSuccess).
The probability to achieve replication success
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
Held, L. (2020). A new standard for the analysis and design of replication studies (with discussion). Journal of the Royal Statistical Society: Series A (Statistics in Society), 183(2), 431-448. doi:10.1111/rssa.12493
## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1) porsPs(level = 0.025, dprior = dprior, sr = c(0.05, 0.01))## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1) porsPs(level = 0.025, dprior = dprior, sr = c(0.05, 0.01))
This function computes the probability to achieve replication success on statistical significance of the replication effect estimate.
porsSig(level, dprior, sr)porsSig(level, dprior, sr)
level |
Significance level for p-value of the replication effect estimate (one-sided and in the same direction as the original effect estimate) |
dprior |
Design prior object |
sr |
Replication standard error |
The probability to achieve replication success
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552
## specify design prior to1 <- 2 so1 <- 1 dprior <- designPrior(to = to1, so = so1, tau = 0.1) porsSig(level = 0.025, dprior = dprior, sr = c(0.5, 0.3))## specify design prior to1 <- 2 so1 <- 1 dprior <- designPrior(to = to1, so = so1, tau = 0.1) porsSig(level = 0.025, dprior = dprior, sr = c(0.5, 0.3))
This function computes the probability to achieve replication success based on establishing the absence of a practically relevant effect size with the Two One-Sided Tests (TOST) procedure in the replication study.
porsTOST(level, dprior, margin, sr)porsTOST(level, dprior, margin, sr)
level |
Significance level for the TOST p-value |
dprior |
Design prior object |
margin |
The equivalence margin > 0 for the equivalence region around zero that defines a region of practically irrelevant effect sizes |
sr |
Replication standard error |
The probability to achieve replication success
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552
Anderson, S. F. and Maxwell, S. E. (2016). There's more than one way to conduct a replication study: Beyond statistical significance. Psychological Methods, 21(1), 1-12. doi:10.1037/met0000051
## specify design prior to1 <- 2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.1) porsTOST(level = 0.1, dprior = dprior, margin = 0.3, sr = c(0.05, 0.03))## specify design prior to1 <- 2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.1) porsTOST(level = 0.1, dprior = dprior, margin = 0.3, sr = c(0.05, 0.03))
"designPrior"
Print method for class "designPrior"
## S3 method for class 'designPrior' print(x, ...)## S3 method for class 'designPrior' print(x, ...)
x |
Object of class |
... |
Other arguments (for consistency with the generic) |
Prints text summary in the console and invisibly returns the
"designPrior" object
Samuel Pawel
dp <- designPrior(to = 0.5, so = 0.05, sp = 0.2, tau = 0.1) print(dp)dp <- designPrior(to = 0.5, so = 0.05, sp = 0.2, tau = 0.1) print(dp)
"ssdRS"
Print method for class "ssdRS"
## S3 method for class 'ssdRS' print(x, ...)## S3 method for class 'ssdRS' print(x, ...)
x |
Object of class |
... |
Other arguments (for consistency with the generic) |
Prints text summary in the console and invisibly returns the
"ssdRS" object
Samuel Pawel
## specify design prior to1 <- 2 so1 <- 1 dprior <- designPrior(to = to1, so = so1) ## compute required standard error for significance at one-sided 2.5% sregionfunSig <- function(sr, alpha = 0.025) { successRegion(intervals = cbind(stats::qnorm(p = 1- alpha)*sr, Inf)) } ssd1 <- ssd(sregionfun = sregionfunSig, dprior = dprior, power = 0.8) print(ssd1)## specify design prior to1 <- 2 so1 <- 1 dprior <- designPrior(to = to1, so = so1) ## compute required standard error for significance at one-sided 2.5% sregionfunSig <- function(sr, alpha = 0.025) { successRegion(intervals = cbind(stats::qnorm(p = 1- alpha)*sr, Inf)) } ssd1 <- ssd(sregionfun = sregionfunSig, dprior = dprior, power = 0.8) print(ssd1)
"successRegion"
Print method for class "successRegion"
## S3 method for class 'successRegion' print(x, ...)## S3 method for class 'successRegion' print(x, ...)
x |
Object of class |
... |
Other arguments |
Prints text summary in the console and invisibly returns the
"successRegion" object
Samuel Pawel
## success region for two-sided significance test successRegion(intervals = rbind(c(1.96, Inf), c(-Inf, -1.96))) ## success region for one-sided significance test successRegion(intervals = rbind(c(1.96, Inf)))## success region for two-sided significance test successRegion(intervals = rbind(c(1.96, Inf), c(-Inf, -1.96))) ## success region for one-sided significance test successRegion(intervals = rbind(c(1.96, Inf)))
This function computes the sample size related to a specified
standard error and unit standard deviation
unitSD, which is the standard deviation of one effective unit (one
measurement, one pair of measurements, one event, etc.). The relationship
is
assumed. The unit standard deviation depends on the parameter type and
the assumptions underlying the standard error calculation. The default is
unitSD = 2 which is, under some assumptions, a reasonable
approximation to the unit standard deviation for standardized mean
differences and log odds/hazard/rate ratios, see Section 2.4 in
Spiegelhalter et al. (2004).
se2n(se, unitSD = 2)se2n(se, unitSD = 2)
se |
Standard error |
unitSD |
Unit standard deviation. Defaults to |
The sample size corresponding to the specified standard error and unit standard deviation
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
Spiegelhalter, D.J., Abrams, K.R., Myles, J.P. (2004). Bayesian approaches to clinical trials and health care evaluation. Wiley. doi:10.1002/0470092602
smd1 <- 0.3 so1 <- 0.05 dprior <- designPrior(to = smd1, so = so1) ssd1 <- ssdSig(level = 0.025, dprior = dprior, power = 0.8) se2n(se = ssd1$sr, unitSD = 2) # required nsmd1 <- 0.3 so1 <- 0.05 dprior <- designPrior(to = smd1, so = so1) ssd1 <- ssdSig(level = 0.025, dprior = dprior, power = 0.8) se2n(se = ssd1$sr, unitSD = 2) # required n
This function computes the standard error of the replication effect estimate required to achieve replication success with a certain probability and based on a certain type of success region.
ssd( sregionfun, dprior, power, nsites = 1, searchInt = c(.Machine$double.eps^0.5, 4), ... )ssd( sregionfun, dprior, power, nsites = 1, searchInt = c(.Machine$double.eps^0.5, 4), ... )
sregionfun |
Function that returns the success region for replication effect estimate as a function of the replication standard error |
dprior |
Design prior object |
power |
Desired probability of replication success |
nsites |
Number of sites. Defaults to |
searchInt |
Search interval for standard errors |
... |
Other arguments passed to |
Returns an object of class "ssdRS" which is a list containing:
designPrior |
The specified "designPrior" object |
power |
The specified power |
powerRecomputed |
The recomputed power |
sr |
The required replication standard error |
c |
The required relative sample size c = nr/no
(assuming so = unitSD/no and sr = unitSD/nr) |
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
## specify design prior to1 <- 2 so1 <- 1 dprior <- designPrior(to = to1, so = so1) ## compute required standard error for significance at one-sided 2.5% sregionfunSig <- function(sr, alpha = 0.025) { successRegion(intervals = cbind(stats::qnorm(p = 1- alpha)*sr, Inf)) } ssd(sregionfun = sregionfunSig, dprior = dprior, power = 0.8)## specify design prior to1 <- 2 so1 <- 1 dprior <- designPrior(to = to1, so = so1) ## compute required standard error for significance at one-sided 2.5% sregionfunSig <- function(sr, alpha = 0.025) { successRegion(intervals = cbind(stats::qnorm(p = 1- alpha)*sr, Inf)) } ssd(sregionfun = sregionfunSig, dprior = dprior, power = 0.8)
This function computes the standard error required to achieve replication success with a certain probability and based on the Bayes factor under normality. The Bayes factor is oriented so that values above one indicate evidence for the null hypothesis of the effect size being zero, whereas values below one indicate evidence for the hypothesis of the effect size being non-zero (with normal prior assigned to it).
ssdBF01( level, dprior, power, priormean = 0, priorvar = 1, searchInt = c(.Machine$double.eps^0.5, 2) )ssdBF01( level, dprior, power, priormean = 0, priorvar = 1, searchInt = c(.Machine$double.eps^0.5, 2) )
level |
Bayes factor level below which replication success is achieved |
dprior |
Design prior object |
power |
Desired probability of replication success |
priormean |
Mean of the normal prior under the alternative. Defaults to
|
priorvar |
Variance of the normal prior under the alternative. Defaults
to |
searchInt |
Interval for numerical search over replication standard errors |
Returns an object of class "ssdRS". See ssd for
details.
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.03) ssdBF01(level = 1/10, dprior = dprior, power = 0.8)## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.03) ssdBF01(level = 1/10, dprior = dprior, power = 0.8)
This function computes the standard error required to achieve replication success with a certain probability and based on the replication Bayes factor under normality. The replication Bayes factor is assumed to be oriented so that values below one indicate replication success, whereas values above one indicate evidence for the null hypothesis.
ssdBFr( level, dprior, power, searchInt = c(.Machine$double.eps^0.5, 2), paradox = TRUE )ssdBFr( level, dprior, power, searchInt = c(.Machine$double.eps^0.5, 2), paradox = TRUE )
level |
Bayes factor level below which replication success is achieved |
dprior |
Design prior object |
power |
Desired probability of replication success |
searchInt |
Interval for numerical search over replication standard errors |
paradox |
Should the probability of replication success be computed
allowing for the replication paradox (replication success when the effect
estimates from original and replication study have a different sign)?
Defaults to |
Returns an object of class "ssdRS". See ssd for
details.
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552
Verhagen, J. and Wagenmakers, E. J. (2014). Bayesian tests to quantify the result of a replication attempt. Journal of Experimental Psychology: General, 145:1457-1475. doi:10.1037/a0036731
Ly, A., Etz, A., Marsman, M., and Wagenmakers, E.-J. (2018). Replication Bayes factors from evidence updating. Behavior Research Methods, 51(6), 2498-2508. doi:10.3758/s13428-018-1092-x
## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.03) ssdBFr(level = 1/10, dprior = dprior, power = 0.8)## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.03) ssdBFr(level = 1/10, dprior = dprior, power = 0.8)
This function computes the standard error required to achieve replication success with a certain probability and based on the sceptical Bayes factor. The sceptical Bayes factor is assumed to be oriented so that values below one indicate replication success.
ssdBFs( level, dprior, power, searchInt = c(.Machine$double.eps^0.5, 2), paradox = TRUE )ssdBFs( level, dprior, power, searchInt = c(.Machine$double.eps^0.5, 2), paradox = TRUE )
level |
Threshold for the sceptical Bayes factor below which replication success is achieved |
dprior |
Design prior object |
power |
Desired probability of replication success |
searchInt |
Interval for numerical search over replication standard errors |
paradox |
Should the probability of replication success be computed
allowing for the replication paradox (replication success when the effect
estimates from original and replication study have a different sign)?
Defaults to |
Returns an object of class "ssdRS". See ssd for
details.
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
Pawel, S. and Held, L. (2020). The sceptical Bayes factor for the assessement of replication success. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 84(3), 879-911. doi:10.1111/rssb.12491
## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.03) ssdBFs(level = 1/10, dprior = dprior, power = 0.9)## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.03) ssdBFs(level = 1/10, dprior = dprior, power = 0.9)
This function computes the standard error required to achieve replication success with a certain probability and based on effect size equivalence of original and replication effect size. Effect size equivalence is defined by the confidence interval for the difference between the original and replication effect sizes falling within an equivalence region around zero defined by the specified margin.
ssdEqu(level, dprior, power, margin, searchInt = c(0, 2))ssdEqu(level, dprior, power, margin, searchInt = c(0, 2))
level |
1 - confidence level of confidence interval for effect size difference |
dprior |
Design prior object |
power |
Desired probability of replication success |
margin |
The equivalence margin > 0 for the symmetric equivalence region around zero |
searchInt |
Interval for numerical search over replication standard errors |
Returns an object of class "ssdRS". See ssd for
details.
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
Anderson, S. F. and Maxwell, S. E. (2016). There's more than one way to conduct a replication study: Beyond statistical significance. Psychological Methods, 21(1), 1-12. doi:10.1037/met0000051
## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.05) ssdEqu(level = 0.1, dprior = dprior, power = 0.8, margin = 0.2)## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.05) ssdEqu(level = 0.1, dprior = dprior, power = 0.8, margin = 0.2)
This function computes the standard error required to achieve replication success with a certain probability and based on statistical significance of the fixed-effect meta-analytic effect estimate obtained from combining original and replication effect estimates.
ssdMeta(level, dprior, power, searchInt = c(0, 10))ssdMeta(level, dprior, power, searchInt = c(0, 10))
level |
Significance level for the replication effect estimate (one-sided and in the same direction as the original effect estimate) |
dprior |
Design prior object |
power |
Desired probability of replication success |
searchInt |
Interval for numerical search over replication standard errors |
Returns an object of class "ssdRS". See ssd for
details.
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
## specify design prior to1 <- 2 so1 <- 1 dprior <- designPrior(to = to1, so = so1, tau = 0.25, sp = Inf) ssdMeta(level = 0.025^2, dprior = dprior, power = 0.95)## specify design prior to1 <- 2 so1 <- 1 dprior <- designPrior(to = to1, so = so1, tau = 0.25, sp = Inf) ssdMeta(level = 0.025^2, dprior = dprior, power = 0.95)
This function computes the standard error required to achieve replication success with a certain probability and based on the sceptical p-value.
ssdPs(level, dprior, power)ssdPs(level, dprior, power)
level |
Threshold for the (one-sided) sceptical p-value below which replication success is achieved |
dprior |
Design prior object |
power |
Desired probability of replication success |
The sceptical p-value is assumed to be uncalibrated as in Held (2020). The package ReplicationSuccess allows for sample size and power calculations with the recalibrated sceptical p-value (https://CRAN.R-project.org/package=ReplicationSuccess).
Returns an object of class "ssdRS". See ssd for
details.
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552
Held, L. (2020). A new standard for the analysis and design of replication studies (with discussion). Journal of the Royal Statistical Society: Series A (Statistics in Society), 183(2), 431-448. doi:10.1111/rssa.12493
## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.03) ssdPs(level = 0.05, dprior = dprior, power = 0.9)## specify design prior to1 <- 0.2 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.03) ssdPs(level = 0.05, dprior = dprior, power = 0.9)
This function computes the standard error required to achieve replication success with a certain probability and based on statistical significance of the replication effect estimate.
ssdSig(level, dprior, power)ssdSig(level, dprior, power)
level |
Significance level for the replication effect estimate (one-sided and in the same direction as the original effect estimate) |
dprior |
Design prior object |
power |
Desired probability of replication success |
Returns an object of class "ssdRS". See ssd for
details.
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552
## specify design prior to1 <- 2 so1 <- 0.5 dprior <- designPrior(to = to1, so = so1, tau = 0.1) ssdSig(level = 0.025, dprior = dprior, power = 0.9)## specify design prior to1 <- 2 so1 <- 0.5 dprior <- designPrior(to = to1, so = so1, tau = 0.1) ssdSig(level = 0.025, dprior = dprior, power = 0.9)
This function computes the standard error required to achieve replication success with a certain probability and based on establishing the absence of a practically relevant effect size with the Two One-Sided Tests (TOST) procedure in the replication study.
ssdTOST(level, dprior, power, margin, searchInt = c(0, 2))ssdTOST(level, dprior, power, margin, searchInt = c(0, 2))
level |
Significance level for the TOST p-value |
dprior |
Design prior object |
power |
Desired probability of replication success |
margin |
The equivalence margin > 0 for the equivalence region around zero that defines a region of practically irrelevant effect sizes |
searchInt |
Interval for numerical search over replication standard errors |
Returns an object of class "ssdRS". See ssd for
details.
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
Anderson, S. F. and Maxwell, S. E. (2016). There's more than one way to conduct a replication study: Beyond statistical significance. Psychological Methods, 21(1), 1-12. doi:10.1037/met0000051
## specify design prior to1 <- 0.05 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.05) ssdTOST(level = 0.05, dprior = dprior, power = 0.9, margin = 0.3)## specify design prior to1 <- 0.05 so1 <- 0.05 dprior <- designPrior(to = to1, so = so1, tau = 0.05) ssdTOST(level = 0.05, dprior = dprior, power = 0.9, margin = 0.3)
Creates a success region object which can then be used for
computing the probability of replication success with pors.
successRegion(intervals)successRegion(intervals)
intervals |
A 2xN matrix containing N disjoint intervals, the first column containing the lower and the second column containing the upper limits |
Returns an object of class "successRegion" which is a matrix
containing the success intervals sorted in ascending order
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
successRegion(intervals = rbind(c(1.96, Inf), c(-Inf, -1.96))) successRegion(intervals = cbind(1.96, Inf))successRegion(intervals = rbind(c(1.96, Inf), c(-Inf, -1.96))) successRegion(intervals = cbind(1.96, Inf))
This function returns the success region for the (meta-analytic) replication effect estimate to achieve significance
successRegionSig(sr, to, tau = 0, nsites = 1, level)successRegionSig(sr, to, tau = 0, nsites = 1, level)
sr |
Replication standard error |
to |
Original effect estimate |
tau |
Heterogeneity standard deviation used in the calculation of the
meta-analytic replication effect estimate and its standard error.
Defaults to |
nsites |
nsites Number of sites, defaults to |
level |
Significance level for p-value of the (average) replication effect estimate (one-sided and in the same direction as the original effect estimate) |
An object of class "successRegion". See
successRegion for details.
Samuel Pawel
Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. doi:10.1037/met0000604
successRegionSig(sr = 0.05, to = 0.2, tau = 0.01, nsites = 3, level = 0.025)successRegionSig(sr = 0.05, to = 0.2, tau = 0.01, nsites = 3, level = 0.025)