`::walkthrough_p() cannonball`

# Walkthrough: A significance test for a two-group comparison

I wrote an R function that’s hopefully useful to teach students what significance tests do and how they can and can’t be interpreted.

**Update (2023-08-06):** By now, this function has long been integrated in the `cannonball`

package. I’ve edited this blog post correspondingly. The installation instructions for the `cannonball`

package are available from its GitHub page.

The basic use is simple. Just run `walkthrough_p()`

and follow the on-screen instructions.

## What does the function do?

Data are generated from a normal distribution with the requested standard deviation. Then, the data points are randomly assigned to two equal-sized groups. Data points in the intervention group receive a boost as specified by `diff`

. Finally, a significance test is run on the data.

By default, the significance test is a two-sample Student’s t-test. Technically, the p-value from this test is the probability that a t-statistic larger than the one observed would’ve been observed if only chance were at play, but the walkthrough text says that is the probability that a mean difference larger than the one observed would’ve been observed if only chance were at play. That is, I use the t-test as an approximation to a permutation test. Switch on pedant mode if you want to run a permutation test.

## Parameters

`n`

: the number of data points per group`diff`

: The boost that participants in the intervention group receive.`sd`

: The standard deviation of the normal distribution from which the data were generated.`showdata`

: Do you want to output a dataframe containing the plotted data (TRUE) or not (FALSE, default)?`pedant`

: Do you want to run the significance test in pedant mode (TRUE; performs a permutation test) or not (FALSE, default; performs Student’s t-test)?

## Examples

```
::walkthrough_p(n = 12, diff = 0.2, sd = 1.3)
cannonball
# Save data and double check results
<- cannonball::walkthrough_p(n = 10, diff = 0.2,
dat sd = 2, showdata = TRUE)
t.test(score ~ group, data = dat, var.equal = TRUE)
# Run in pedant mode (= permutation test)
<- cannonball::walkthrough_p(n = 13, diff = 1, sd = 4,
dat pedant = TRUE, showdata = TRUE)
t.test(score ~ group, data = dat, var.equal = TRUE)
```