DesignLibrary 2.0 replaces the designer functions of 0.1 with a
library of self-contained design files, read through
make_design(). It requires DeclareDesign, fabricatr,
estimatr, and randomizr 2.0.
All fifteen 0.1 designers keep their names and their argument names,
with two exceptions: block_cluster_two_arm_designer() loses
six arguments, and three arguments (label_E1 and
label_E2 of process_tracing_designer(), and
args_to_fix of every designer) are accepted but ignored
with a warning. Each now builds a library design, so
two_arm_designer(N = 40) returns the same design as
make_design("two_arm_flexible", N = 40). Most 0.1 scripts
that call a designer and diagnose the result run unchanged.
There are two types of break:
Removed functions and removed arguments stop with an error that names the replacement.
A handful of changes do not error. These aries if there is a changed default, renamed inquiries, or renamed estimator terms or different step names. See Porting an existing script for more.
A 0.1 script like this one runs as written. As in 0.1,
library(DesignLibrary) also attaches DeclareDesign,
fabricatr, estimatr, and randomizr.
library(DesignLibrary)
design <- two_arm_designer(N = 100, ate = 0.25)
set.seed(343)
draw_estimates(design)
#> # A tibble: 1 × 12
#> term estimate std.error statistic p.value conf.low conf.high df outcome
#> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <chr>
#> 1 Z -0.0159 0.171 -0.0929 0.926 -0.356 0.324 98 Y
#> # ℹ 3 more variables: estimator <chr>, inquiry <chr>, estimand <dbl>What carries over:
two_arm_designer(),
two_arm_attrition_designer(),
two_arm_covariate_designer(),
pretest_posttest_designer(),
randomized_response_designer(),
mediation_analysis_designer(),
multi_arm_designer(), two_by_two_designer(),
factorial_designer(),
block_cluster_two_arm_designer(),
regression_discontinuity_designer(),
spillover_designer(),
cluster_sampling_designer(),
binary_iv_designer(), and
process_tracing_designer().block_cluster_two_arm_designer() and the default
N of multi_arm_designer().two_arm_covariate,
regression_discontinuity_polynomial,
spillover, cluster_sampling,
binary_iv, process_tracing_bayes, and
factorial) flattened from their 0.1 code, and at the same
arguments and seed each draws what its 0.1 designer drew.Four aliases that 0.1 had already deprecated are gone, along with two helpers. In each case you get an informative message.
simple_two_arm_designer(N = 50)
#> Error in `.defunct_01()`:
#> ! `simple_two_arm_designer()` was removed in DesignLibrary 2.0.0.
#> ℹ Use `two_arm_designer()`, which takes the same arguments.| Removed | Use instead |
|---|---|
simple_two_arm_designer() |
two_arm_designer(), same arguments |
simple_factorial_designer() |
two_by_two_designer(), same arguments |
simple_iv_designer() |
binary_iv_designer(), same arguments |
simple_spillover_designer() |
spillover_designer(), same arguments |
get_design_code() |
get_code("<id>") |
match.call.defaults() |
match.call() and formals() |
In 0.1 every designer pasted its own source code into an attribute of
the design it returned, and get_design_code() read that
attribute back. In 2.0 the source of a design is a file in the package,
so there is nothing to paste: get_code() prints the file.
Each designer’s help page names the id of the file it builds.
get_design_code(two_arm_designer())
#> Error in `.defunct_01()`:
#> ! `get_design_code()` was removed in DesignLibrary 2.0.0.
#> ℹ Designs no longer carry their code as an attribute.
#> ℹ Use `get_code()` with a library design id, for example `get_code("two_arm_flexible")`.
#> ℹ `list_designs()` lists the ids, and each designer's help page names the id it builds.get_code("two_arm_flexible")
#> design <-
#> declare_parameters(
#> N = 100,
#> assignment_prob = 0.5,
#> control_mean = 0,
#> control_sd = 1,
#> ate = 1,
#> treatment_sd = 1,
#> rho = 1
#> ) +
#> declare_model(
#> N = N,
#> u_0 = rnorm(N),
#> u_1 = rnorm(n = N, mean = rho * u_0, sd = sqrt(pmax(0, 1 - rho^2))),
#> potential_outcomes(
#> Y ~ control_mean +
#> (1 - Z) * (u_0 * control_sd) + Z* (u_1 * treatment_sd + ate)
#> )
#> ) +
#> declare_inquiry(ATE = mean(Y_Z_1 - Y_Z_0)) +
#> declare_assignment(Z = complete_ra(N, prob = assignment_prob)) +
#> declare_measurement(Y = reveal_outcomes(Y ~ Z)) +
#> declare_estimator(Y ~ Z, inquiry = "ATE")The description, definitions, and
shiny_arguments attributes went with code. Now
get_args() and design_info() describe a
design’s parameters.
block_cluster_two_arm_designer() takes new
argumentsThe 0.1 designer derived its level sizes from N and its
three shock standard deviations from one total sd, through
defaults that referred to each other. Its defaults also produced a
design with no blocking and no clustering (one block of 100 single-unit
clusters) and no effect. The 2.0 designer takes the level sizes and the
shocks directly, and its defaults are a blocked and clustered design
with an effect to find: 20 blocks of 4 clusters of 10 units, each shock
with standard deviation 0.577, and ate = 0.2.
The 0.1 arguments N, sd,
sd_i_1, rho, control_mean, and
treatment_mean are errors that say what to set instead.
block_cluster_two_arm_designer(N = 200, rho = 0.5)
#> Error in `block_cluster_two_arm_designer()`:
#> ! `block_cluster_two_arm_designer()` no longer takes `N`, `rho` (DesignLibrary 0.1).
#> ℹ Set the sample size through `N_blocks`, `N_clusters_in_block`, and `N_i_in_cluster`.
#> ℹ Treatment and control potential outcomes are perfectly correlated; `rho` is not supported.design <- block_cluster_two_arm_designer(
N_blocks = 5, N_clusters_in_block = 4, N_i_in_cluster = 10, ate = 0.1
)assignment_probs takes one probability rather than one
per block, and verbose is ignored with a warning.
Because the defaults changed,
block_cluster_two_arm_designer() with no arguments is a
different design in 2.0, and nothing warns about it.
multi_arm_designer() has a larger default sampleAs in 0.1, the designer declares an inquiry for every pair of arms
and estimates each by a difference in means between its two arms, under
labels like "DIM (Z_2 - Z_1)". The default N
went from 30 to 90, so that each of the three arms has 30 units rather
than 10.
design <- multi_arm_designer(m_arms = 3)
set.seed(343)
draw_estimates(design)[, c("estimator", "term", "estimate", "inquiry")]
#> # A tibble: 3 × 4
#> estimator term estimate inquiry
#> <chr> <chr> <dbl> <chr>
#> 1 DIM (Z_2 - Z_1) Z2 -0.621 ate_Y_2_1
#> 2 DIM (Z_3 - Z_1) Z3 -0.350 ate_Y_3_1
#> 3 DIM (Z_3 - Z_2) Z3 0.271 ate_Y_3_2mediation_analysis_designer()’s inquiries
Indirect_0 and Indirect_1 are now
Mediator_Effect_0 and Mediator_Effect_1. Each
is the effect of the mediator on the outcome with treatment held at 0 or
at 1.
draw_estimands(mediation_analysis_designer())
#> # A tibble: 7 × 2
#> inquiry estimand
#> <chr> <dbl>
#> 1 FirstStage 0.335
#> 2 Mediator_Effect_0 0.4
#> 3 Mediator_Effect_1 0.4
#> 4 Controlled_Direct_0 0.5
#> 5 Controlled_Direct_1 0.5
#> 6 Natural_Direct_0 0.5
#> 7 Natural_Direct_1 0.5process_tracing_designer() fixes its estimator labels at
"Straw in the Wind" and "Smoking Gun".
args_to_fix is ignored0.1 designers could fix arguments into the returned design so that
expand_design() could not vary them. In 2.0 every parameter
stays redesignable on the design object, and a non-empty
args_to_fix warns.
DeclareDesign 2.0 has no separate potential outcomes or reveal steps:
potential outcomes live in declare_model() and are revealed
in declare_measurement(). Some step names differ as a
result. factorial_designer()’s inquiry step, for example,
was "ATE" and is now "inquiry".
names(factorial_designer())
#> [1] "parameters" "model" "assignment" "measurement" "inquiry"
#> [6] "estimator"Inquiry and estimator labels are otherwise those of 0.1. Code
that picks a step by name or by position (design[[3]])
needs checking, since the position of a step can change even
where its name does not.
expand_design() still works but is deprecated in
DeclareDesign 2.0. Pass a vector to redesign() instead,
which needs no designer function at all.
One caution: process_tracing_designer() checks that
cor_E1E2_H and cor_E1E2_not_H are compatible
with the clue probabilities, and redesign() does not. An
incompatible value passed through redesign() fails only
when the design is run, with “negative probability” from the measurement
step.
Search the script for each pattern in the left column.
| Search for | Replace with | If you do not |
|---|---|---|
simple_two_arm_designer,
simple_factorial_designer, simple_iv_designer,
simple_spillover_designer |
two_arm_designer, two_by_two_designer,
binary_iv_designer, spillover_designer |
error naming the replacement |
get_design_code( |
get_code("<id>") |
error naming the replacement |
match.call.defaults( |
match.call() and formals() |
error |
block_cluster_two_arm_designer( with N,
sd, sd_i_1, rho,
control_mean, or treatment_mean |
N_blocks, N_clusters_in_block,
N_i_in_cluster, sd_block,
sd_cluster, sd_i, ate |
error naming the replacement |
block_cluster_two_arm_designer() relying on
defaults |
state the 0.1 sizes and ate = 0 explicitly |
silently a different design |
multi_arm_designer( without N |
N = 30 for the 0.1 default |
silently a larger sample |
Indirect_0, Indirect_1 |
Mediator_Effect_0, Mediator_Effect_1 |
filter returns no rows |
design[[ with a number, or step names such as
"ATE" |
names(design) to find the step |
wrong step, or error |
args_to_fix |
drop it | warning |
expand_design( |
redesign(designer(), arg = values) |
deprecation warning |
make_design(design = |
make_design("<id>") |
error |
The rows in bold are the ones to search for deliberately, since nothing will point you to them.
The big change is that the key files contained in the library are
simple design scripts rather than “designer functions.” This makes it
much easier to inspect a design and to contribute a design. In addition
the designs make use of the the new declare_parameters
functionality from DeclareDesign 2.0 which makes it easier to contribute
modifiable designs.
The library holds 75 designs, including the declarations from Research Design in the Social Sciences.
list_designs() browses the librarymake_design() builds a design and changes its
parametersget_args() lists what can be changedget_code() prints the sourcedesigns_shiny_app() opens the library in a browservignette("DesignLibrary") gives you a tour.