Welcome to PyPowerUp’s documentation!

Authors

Sophia Man Yang, Nianbo Dong, Rebecca Maynard

PyPowerUp

PyPowerUp is the Python implementation for the research article “PowerUp!: A Tool for Calculating Minimum Detectable Effect Sizes and Minimum Required Sample Sizes for Experimental and Quasi-experimental Design Studies (Dong & Maynard, 2013)”. It is a power analysis tool for 21 experimental and quasi-experimental designs.

Given study design, PyPowerUp computes minimum detectable effect sizes effect_size, power power, and minimum required samples sizes sample size.

To install PyPowerUp, run this command in your terminal:

$ pip install pypowerup

To use the functions:

from pypowerup import effect_size, power, sample_size

Individual Random Assignment Designs

Simple Individual Random Assignment (IRA)

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
simple	1	individual	1	NA

from pypowerup import effect_size, power, sample_size

# effect size, i.e., minimum detectable effect sizes (MDES):
effect_size(design='ira', n=787, power=0.8, alpha=0.05, two_tailed=True, p=0.5, r21=0, g=0)

0.19997988869985736

# sample size, i.e., minimum required samples sizes (MRSS):
sample_size(design='ira', es=0.2, power=0.8, alpha=0.05, two_tailed=True, p=0.5, r21=0, g=0)

787.0

# power
power(design='ira', es=0.2, n=787, alpha=0.05, two_tailed=True, p=0.5, r21=0, g=0)

0.8000799952735076

Parameters for IRA

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
g	✓	✓	✓

Output validation with statsmodels

Here we used statsmodels with the same parameters to validate our model results. Note nobs1 in statsmodels is the sample size for treatment group, which is half of the total sample size output from our model. Thus, we define nobs1=787/2=393.5 in the models beflow.

from statsmodels.stats.power import TTestIndPower

# effect size
analysis = TTestIndPower()
analysis.solve_power(power=0.8, nobs1=393.5, ratio=1, alpha=0.05, alternative='two-sided')

0.19997768751017836

# sample size (statsmodels output treatment sample size only, which is the half of our computed total sample size)
analysis.solve_power(0.2, power=0.8, nobs1=None, ratio=1, alpha=0.05,alternative='two-sided')

393.4056989990335

# power
analysis.solve_power(0.2, nobs1=393.5, ratio=1.0, alpha=0.05, alternative='two-sided')

0.8000942129794306

With the same parameters, we get the same results as the statsmodels. However, statsmodels does not do power analysis for more complicated designs, which we will cover next.

Blocked (Stratified) Individual Random Assignment (BIRA)

BIRA2_1c

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	2	individual	1	constant

from pypowerup import effect_size, power, sample_size

# effect size, i.e., minimum detectable effect sizes (MDES):
effect_size(design = "bira2_1c", n=80, J=14, power=0.8, alpha=0.05, two_tailed=True, p=0.5, r21=0, g=0)

0.1675746241777643

# sample_size, i.e., minimum required samples sizes (MRSS) for level 2 units.
sample_size(design = "bira2_1c", n=80, es=0.168, power=0.8, alpha=0.05, two_tailed=True, p=0.5, r21=0, g=0)

14.0

# power
power(design = "bira2_1c", n=80, J=14, es=0.168, alpha=0.05, two_tailed=True, p=0.5, r21=0, g=0)

0.801986386400488

Parameters for BIRA2_1c:

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
g	✓	✓	✓

BIRA2_1f

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	2	individual	1	fixed

# effect size, i.e., minimum detectable effect sizes (MDES):
effect_size(design = "bira2_1f", n=10, J=200, power=0.8, alpha=0.05, two_tailed=True, p=0.5, r21=0, g=0)

0.1253671054395026

# sample_size, i.e., minimum required samples sizes (MRSS) for level 2 units.
sample_size(design = "bira2_1f", n=10, es=0.125, power=0.8, alpha=0.05, two_tailed=True, p=0.5, r21=0, g=0)

201.0

# power
power(design = "bira2_1f", n=10, J=200, es=0.125, alpha=0.05, two_tailed=True, p=0.5, r21=0, g=0)

0.7976961337604935

Parameters for BIRA2_1f

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
g	✓	✓	✓

BIRA2_1r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	2	individual	1	random

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design = "bira2_1r", n=80, J=480, rho2=0.35, omega2=0.1, power=0.8, alpha=0.05,
            two_tailed=True, p=0.5, r21=0, r2t2=0, g=0)

0.033290496240165834

# sample_size, i.e., minimum required samples sizes (MRSS) for level 2 units
sample_size(design = "bira2_1r", es=0.033290496240165834, n=80, rho2=0.35, omega2=0.1, power=0.8, alpha=0.05,
            two_tailed=True, p=0.5, r21=0, r2t2=0, g=0)

480.0

# power
power(design = "bira2_1r", es=0.033290496240165834, n=80, J=480, rho2=0.35, omega2=0.1, alpha=0.05,
      two_tailed=True, p=0.5, r21=0, r2t2=0, g=0)

0.800001211704756

Parameters for BIRA2_1r

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
g	✓	✓	✓
rho2	✓	✓	✓
omega2	✓	✓	✓
r2t2	✓	✓	✓

BIRA3_1r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	3	individual	1	random

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design = "bira3_1r", n=80, J=10, K=100, rho3=0.2, rho2=0.15, omega3=0.1, omega2=0.1,
            power=0.8, alpha=0.05, two_tailed=True, p=0.5, r21=0, r2t2=0, r2t3=0, g=0)

0.04451381756409839

# sample_size, i.e., minimum required samples sizes (MRSS) for level 3 units
sample_size(design = "bira3_1r", es=0.04451381756409839, n=80, J=10, rho3=0.2, rho2=0.15, omega3=0.1, omega2=0.1,
            power=0.8, alpha=0.05, two_tailed=True, p=0.5, r21=0, r2t2=0, r2t3=0, g=0)

100.0

# power
power(design = "bira3_1r", es=0.04451381756409839, n=80, J=10, K=100, rho3=0.2, rho2=0.15, omega3=0.1, omega2=0.1,
            alpha=0.05, two_tailed=True, p=0.5, r21=0, r2t2=0, r2t3=0, g=0)

0.8000026614774113

Parameters for BIRA3_1r

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓	✓	✓
K	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
g	✓	✓	✓
rho2	✓	✓	✓
omega2	✓	✓	✓
r2t2	✓	✓	✓
rho3	✓	✓	✓
omega3	✓	✓	✓
r2t3	✓	✓	✓

BIRA4_1r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	4	individual	1	random

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design = "bira4_1r", n=10, J=4, K=4, L=20, rho4=0.05, rho3=0.15, rho2=0.15,
            omega4=0.5, omega3=0.5, omega2=0.5, r21=0.5, r2t2=0.5, r2t3=0.5, r2t4=0.5, g=1,
            power=0.8, alpha=0.05, two_tailed=True
           )

0.1191537036343213

# sample_size, i.e., minimum required samples sizes (MRSS) for level 4 units
sample_size(design = "bira4_1r", es=0.1191537036343213, n=10, J=4, K=4, rho4=0.05, rho3=0.15, rho2=0.15,
            omega4=0.5, omega3=0.5, omega2=0.5, r21=0.5, r2t2=0.5, r2t3=0.5, r2t4=0.5, g=1,
            power=0.8, alpha=0.05, two_tailed=True)

20.0

# power
power(design = "bira4_1r", es=0.1191537036343213, n=10, J=4, K=4, L=20, rho4=0.05, rho3=0.15, rho2=0.15,
            omega4=0.5, omega3=0.5, omega2=0.5, r21=0.5, r2t2=0.5, r2t3=0.5, r2t4=0.5, g=1,
            alpha=0.05, two_tailed=True)

0.8000404338335202

Parameters for BIRA4_1r

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓	✓	✓
K	✓	✓	✓
L	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
g	✓	✓	✓
rho2	✓	✓	✓
omega2	✓	✓	✓
r2t2	✓	✓	✓
rho3	✓	✓	✓
omega3	✓	✓	✓
r2t3	✓	✓	✓
rho4	✓	✓	✓
omega4	✓	✓	✓
r2t4	✓	✓	✓

Cluster Random Assignment Designs

Simple Cluster Random Assignment (CRA)

CRA2_2r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
simple	2	cluster	2	random

from pypowerup import effect_size, sample_size, power

# effect size, i.e., minimum detectable effect sizes (MDES):
effect_size(design='cra2_2r', rho2=0.15, r21=0.40, r22=0.53, g=1, n=100, J=40,
           alpha=0.05, power=0.8, p=0.5)

0.2502054873913746

# sample_size, i.e., minimum required samples sizes (MRSS) for level 2 units.
sample_size(design='cra2_2r', es=0.2502054873913746, rho2=0.15, r21=0.40, r22=0.53,
            g=1, n=100, alpha=0.05, power=0.8, p=0.5)

40.0

# power
power(design='cra2_2r', es=0.2502054873913746, rho2=0.15, r21=0.40,
      r22=0.53, g=1, n=100, J=40, alpha=0.05, p=0.5)

0.8000094900703394

Parameters for CRA2_2r

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
r22	✓	✓	✓
rho2	✓	✓	✓
g	✓	✓	✓

CRA3_3r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
simple	3	cluster	3	random

# effect size, i.e., minimum detectable effect sizes (MDES):
effect_size(design='cra3_3r', rho3=0.38, rho2=0.10, r21=0.37, r22=0.53,
            r23=0.87, g=1, n=20, J=2, K=66, alpha=0.05, power=0.8, p=0.5)

0.19949475029929953

# sample_size, i.e., minimum required samples sizes (MRSS) for level 3 units.
sample_size(design='cra3_3r', es=0.19949475029929953, rho3=0.38,
            rho2=0.10, r21=0.37, r22=0.53, r23=0.87, g=1, n=20, J=2,
            alpha=0.05, power=0.8, p=0.5)

66.0

# power
power(design='cra3_3r', es=0.19949475029929953, rho3=0.38, rho2=0.10, r21=0.37,
      r22=0.53, r23=0.87, g=1, n=20, J=2, K=66, alpha=0.05, p=0.5)

0.8000043161361032

Parameters for CRA3_3r

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓	✓	✓
K	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
r22	✓	✓	✓
rho2	✓	✓	✓
r23	✓	✓	✓
rho3	✓	✓	✓
g	✓	✓	✓

CRA4_4r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
simple	4	cluster	4	random

# effect size, i.e., minimum detectable effect sizes (MDES):
effect_size(design='cra4_4r', rho4=0.05, rho3=0.05, rho2=0.10, r21=0.50, r22=0.50,
            r23=0.50, r24=0.50, g=1, n=10, J=2, K=3, L=20,
            alpha=0.05, power=0.8, p=0.5)

0.29231230598875796

# sample_size, i.e., minimum required samples sizes (MRSS) for level 4 units.
sample_size(design='cra4_4r', es=0.29231230598875796, rho4=0.05, rho3=0.05, rho2=0.10, r21=0.50, r22=0.50,
            r23=0.50, r24=0.50, g=1, n=10, J=2, K=3, alpha=0.05, power=0.8, p=0.5)

20.0

# power
power(design='cra4_4r', es=0.29231230598875796, rho4=0.05, rho3=0.05, rho2=0.10, r21=0.50, r22=0.50,
            r23=0.50, r24=0.50, g=1, n=10, J=2, K=3, L=20, alpha=0.05, p=0.5)

0.800046080238779

Parameters for CRA4_4r

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓	✓	✓
K	✓	✓	✓
L	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
r22	✓	✓	✓
rho2	✓	✓	✓
r23	✓	✓	✓
rho3	✓	✓	✓
r24	✓	✓	✓
rho4	✓	✓	✓
g	✓	✓	✓

Blocked (Stratified) Cluster Randomized Assignment (BCRA)

BCRA3_2f

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	3	cluster	2	fixed

from pypowerup import effect_size, sample_size, power

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design='bcra3_2f', rho2=0.10, r21=0.50, r22=0.50, g=1, n=20, J=44, K=5)

0.10219354337360606

# sample_size, i.e., minimum required samples sizes (MRSS) for level 3 units
sample_size(design='bcra3_2f', es=0.10219354337360606, rho2=0.10, r21=0.50, r22=0.50, g=1, n=20, J=44)

5.0

# power
power(design='bcra3_2f', es=0.10219354337360606, rho2=0.10, r21=0.50, r22=0.50, g=1, n=20, J=44, K=5)

0.8000016066584502

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓	✓	✓
K	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
rho2	✓	✓	✓
r22	✓	✓	✓
g	✓	✓	✓

BCRA3_2r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	3	cluster	2	random

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design='bcra3_2r', rho3=0.38, rho2=0.10, omega3=0.50, r21=0.37,
            r22=0.53, r2t3=0, g=0, n=20, J=2, K=64)

0.20020041517111645

# sample_size, i.e., minimum required samples sizes (MRSS) for level 3 units
sample_size(design='bcra3_2r', es=0.20020041517111645, rho3=0.38, rho2=0.10, omega3=0.50, r21=0.37,
            r22=0.53, r2t3=0, g=0, n=20, J=2)

64.0

# power
power(design='bcra3_2r', es=0.20020041517111645, rho3=0.38, rho2=0.10, omega3=0.50, r21=0.37,
            r22=0.53, r2t3=0, g=0, n=20, J=2, K=64)

0.8000043161361032

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓	✓	✓
K	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
rho2	✓	✓	✓
r22	✓	✓	✓
r2t3	✓	✓	✓
rho3	✓	✓	✓
omega3	✓	✓	✓
g	✓	✓	✓

BCRA4_2r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	4	cluster	2	random

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design='bcra4_2r', rho4=0.05, rho3=0.15, rho2=0.15, omega4=0.5, omega3=0.5, r21=0.5, r22=0.5,
            r2t3=0.5, r2t4=0.5, g=0, n=10, J=4, K=4, L=20)

0.14584081061169768

# sample_size, i.e., minimum required samples sizes (MRSS) for level 4 units
sample_size(design='bcra4_2r', es=0.14584081061169768, rho4=0.05, rho3=0.15, rho2=0.15, omega4=0.5, omega3=0.5, r21=0.5, r22=0.5,
            r2t3=0.5, r2t4=0.5, g=0, n=10, J=4, K=4)

20.0

# power
power(design='bcra4_2r', es=0.14584081061169768, rho4=0.05, rho3=0.15, rho2=0.15, omega4=0.5, omega3=0.5, r21=0.5, r22=0.5,
            r2t3=0.5, r2t4=0.5, g=0, n=10, J=4, K=4, L=20)

0.8000357933435768

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓	✓	✓
K	✓	✓	✓
L	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
rho2	✓	✓	✓
r22	✓	✓	✓
rho3	✓	✓	✓
omega3	✓	✓	✓
r2t3	✓	✓	✓
rho4	✓	✓	✓
omega4	✓	✓	✓
r2t4	✓	✓	✓
g	✓	✓	✓

BCRA4_3f

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	4	cluster	3	fixed

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design='bcra4_3f', rho3=0.15, rho2=0.15, r21=0.5, r22=0.5,
            r23=0.5, g=2, n=10, J=4, K=4, L=15)

0.2399780453218905

# sample_size, i.e., minimum required samples sizes (MRSS) for level 4 units
sample_size(design='bcra4_3f', es=0.2399780453218905, rho3=0.15, rho2=0.15, r21=0.5, r22=0.5,
            r23=0.5, g=2, n=10, J=4, K=4)

15.0

# power
power(design='bcra4_3f', es=0.2399780453218905, rho3=0.15, rho2=0.15, r21=0.5, r22=0.5,
            r23=0.5, g=2, n=10, J=4, K=4, L=15)

0.8000158353898167

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓	✓	✓
K	✓	✓	✓
L	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
rho2	✓	✓	✓
r22	✓	✓	✓
rho3	✓	✓	✓
r23	✓	✓	✓
g	✓	✓	✓

BCRA4_3r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	4	cluster	3	random

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design='bcra4_3r', rho4=0.05, rho3=0.15, rho2=0.15, omega4=0.5,
            r21=0.5, r22=0.5, r23=0.5, r2t4=0.5, g=3, n=10, J=4, K=20, L=20)

0.12100407246925271

# sample_size, i.e., minimum required samples sizes (MRSS) for level 4 units
sample_size(design='bcra4_3r', es=0.12100407246925271, rho4=0.05, rho3=0.15, rho2=0.15, omega4=0.5,
            r21=0.5, r22=0.5, r23=0.5, r2t4=0.5, g=3, n=10, J=4, K=20)

20.0

# power
power(design='bcra4_3r', es=0.12100407246925271, rho4=0.05, rho3=0.15, rho2=0.15, omega4=0.5,
            r21=0.5, r22=0.5, r23=0.5, r2t4=0.5, g=3, n=10, J=4, K=20, L=20)

0.800053034680572

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓	✓	✓
K	✓	✓	✓
L	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
rho2	✓	✓	✓
r22	✓	✓	✓
rho3	✓	✓	✓
r23	✓	✓	✓
rho4	✓	✓	✓
omega4	✓	✓	✓
r2t4	✓	✓	✓
g	✓	✓	✓

Quasi-experimental Designs

Regression Discontinuity (RD)

RD2_1f

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	2	individual	1	fixed

from pypowerup import effect_size, sample_size, power

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design = "rd2_1f", n=55, J=20, r21=0.5, g=1, design_effect=2.75)

0.19828457764454652

# sample_size, i.e., minimum required samples sizes (MRSS) for level 2 units
sample_size(design = "rd2_1f", es=0.19828457764454652, n=55, r21=0.5, g=1, design_effect=2.75)

20.0

# power
power(design = "rd2_1f", es=0.19828457764454652, n=55, r21=0.5, J=20, g=1, design_effect=2.75)

0.8000010686602212

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
g	✓	✓	✓
design_effect	✓	✓	✓

RD2_1r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	2	individual	1	random

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design = "rd2_1r", n=50, J=40, r21=0.5, g=1, r2t2=0.1, omega2=0.2, rho2= 0.15, design_effect=2.75)

0.15782962225367275

# sample_size, i.e., minimum required samples sizes (MRSS) for level 2 units
sample_size(design = "rd2_1r", es=0.15782962225367275, n=50,r21=0.5, g=1,
          r2t2=0.1, omega2=0.2, rho2= 0.15, design_effect=2.75)

40.0

# power
power(design = "rd2_1r", es=0.15782962225367275, n=50, J=40, r21=0.5, g=1,
      r2t2=0.1, omega2=0.2, rho2= 0.15, design_effect=2.75)

0.8000090684884187

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
rho2	✓	✓	✓
r2t2	✓	✓	✓
g	✓	✓	✓
design_effect	✓	✓	✓

RDC_2r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
simple	2	cluster	2	random

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design = "rdc_2r", rho2=0.15, r21=0.5, r22=0.5, g=1, n=55, J=179, design_effect=2.75)

0.20086870136611698

# sample_size, i.e., minimum required samples sizes (MRSS) for level 2 units
sample_size(design = "rdc_2r", es=0.20086870136611698, rho2=0.15, r21=0.5,
            r22=0.5, g=1, n=55, design_effect=2.75)

179.0

# power
power(design = "rdc_2r", es=0.20086870136611698, rho2=0.15, r21=0.5,
            r22=0.5, g=1, n=55, J=179, design_effect=2.75)

0.8000017574724924

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
rho2	✓	✓	✓
r22	✓	✓	✓
g	✓	✓	✓
design_effect	✓	✓	✓

RDC_3r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
simple	3	cluster	3	random

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design = "rdc_3r", rho3=0.15, rho2=0.15, r21=0.5, r22=0.5,
            r23=0.5,  g=1, n=18, J=3, K=230, design_effect=2.75)

0.20079075638849297

# sample_size, i.e., minimum required samples sizes (MRSS) for level 2 units
sample_size(design = "rdc_3r", es=0.20079075638849297, rho3=0.15, rho2=0.15, r21=0.5, r22=0.5,
            r23=0.5,  g=1, n=18, J=3, design_effect=2.75)

230.0

# power
power(design = "rdc_3r", es=0.20079075638849297, rho3=0.15, rho2=0.15, r21=0.5, r22=0.5,
            r23=0.5,  g=1, n=18, J=3, K=230, design_effect=2.75)

0.8000015459112233

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓	✓	✓
K	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
rho2	✓	✓	✓
r22	✓	✓	✓
rho3	✓	✓	✓
r23	✓	✓	✓
g	✓	✓	✓
design_effect	✓	✓	✓

RD2_3f

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	3	cluster	2	fixed

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design = "rd3_2f", rho2=0.15, r21=0.5, r22=0.5, g=0, n=18, J=3, K=71, design_effect=2.75)

0.20131125779908843

# sample_size, i.e., minimum required samples sizes (MRSS) for level 3 units
sample_size(design = "rd3_2f", es=0.20131125779908843, rho2=0.15, r21=0.5, r22=0.5, g=0,
            n=18, J=3, design_effect=2.75)

71.0

# power
power(design = "rd3_2f", es=0.20131125779908843, rho2=0.15, r21=0.5, r22=0.5, g=0,
            n=18, J=3, K=71, design_effect=2.75)

0.8000020062425997

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓	✓	✓
K	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
rho2	✓	✓	✓
r22	✓	✓	✓
g	✓	✓	✓
design_effect	✓	✓	✓

RD2_3r

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	3	cluster	2	random

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design = "rdc_3r", rho3=0.15, rho2=0.15, r21=0.5, r22=0.5, r23=0.5,
            g=1, n=18, J=3, K=230, design_effect=2.75)

0.20079075638849297

# sample_size, i.e., minimum required samples sizes (MRSS) for level 3 units
sample_size(design = "rdc_3r", es=0.20079075638849297, rho3=0.15, rho2=0.15, r21=0.5, r22=0.5, r23=0.5,
            g=1, n=18, J=3, design_effect=2.75)

230.0

# power
power(design = "rdc_3r", es=0.20079075638849297, rho3=0.15, rho2=0.15, r21=0.5, r22=0.5, r23=0.5,
            g=1, n=18, J=3, K=230, design_effect=2.75)

0.8000015459112233

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
J	✓	✓	✓
K	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r21	✓	✓	✓
rho2	✓	✓	✓
r22	✓	✓	✓
rho3	✓	✓	✓
r2t3	✓	✓	✓
omega3	✓	✓	✓
g	✓	✓	✓
design_effect	✓	✓	✓

Interrupted Time-Series (ITS)

Assignment	Clustering Level	Treatment Assignment	Treatment Level	Cluster Effect
blocked	3	cluster	2	constant at level 3 random at level 2

from pypowerup import effect_size, sample_size, power

No comparison units

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design = "its_nocompare", rho2=0.03, T=5, n=75, K=10, r22=0, tf=2, g=0)

0.3658177052373508

# sample_size, i.e., minimum required samples sizes (MRSS) for level 3 units
sample_size(design = "its_nocompare", es=0.3658177052373508, rho2=0.03, T=5, n=75, r22=0, tf=2, g=0)

10.0

# power
power(design = "its_nocompare", es=0.3658177052373508, rho2=0.03, T=5, n=75, K=10, r22=0, tf=2, g=0)

0.8000060699483386

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
K	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r22	✓	✓	✓
rho2	✓	✓	✓
g	✓	✓	✓
T	✓	✓	✓
tf	✓	✓	✓

With comparison units

# effect size, i.e., minimum detectable effect sizes (MDES)
effect_size(design = "its_wcompare", rho2=0.03, T=5, n=75, K=10, r22=0, tf=2, g=0,q=2)

0.44803335835368546

# sample_size, i.e., minimum required samples sizes (MRSS) for level 3 units
sample_size(design = "its_wcompare", es=0.44803335835368546, rho2=0.03, T=5, n=75, r22=0, tf=2, g=0, q=2)

10.0

# power
power(design = "its_wcompare", es=0.44803335835368546, rho2=0.03, T=5, n=75, K=10, r22=0, tf=2, g=0, q=2)

0.8000060699483386

Parameters	effect_size	sample_size	power
design	✓	✓	✓
es		✓	✓
n	✓	✓	✓
K	✓		✓
power	✓	✓
alpha	✓	✓	✓
two_tailed	✓	✓	✓
p	✓	✓	✓
r22	✓	✓	✓
rho2	✓	✓	✓
g	✓	✓	✓
T	✓	✓	✓
tf	✓	✓	✓
q	✓	✓	✓

Credit and disclaimer

This document is heavily built on https://www.causalevaluation.org/uploads/7/3/3/6/73366257/powerup.xlsm. All the design and variable explanations are from this sheet.

References

Dong, N. & Maynard, R. A. (2013). PowerUp!: A tool for calculating minimum detectable effect sizes and minimum required sample sizes for experimental and quasi- experimental design studies, Journal of Research on Educational Effectiveness, 6(1), 24-67. doi: 10.1080/19345747.2012.673143. https://www.causalevaluation.org/uploads/7/3/3/6/73366257/powerup.xlsm

Bulus, M., Dong, N., Kelcey, B., & Spybrook, J. (2019). PowerUpR: Power Analysis Tools for Multilevel Randomized Experiments. R package version 1.0.4. https://CRAN.R-project.org/package=PowerUpR