# Walkthrough: A significance test for a two-group comparison

I wrote an R function that’s hopefully useful to teach students what significance tests do and how they can and can’t be interpreted.

I’ll integrate the function in the `cannonball`

package when I’ll come round to it. In the meantime,
you can use it by running the following line.

The basic use is simple. Just run `walkthrough_p()`

and follow
the on-screen instructions.

## What does the function do?

Data are generated from a normal distribution with the requested standard deviation. Then, the data points are randomly assigned to two equal-sized groups. Data points in the intervention group receive a boost as specified by ‘diff’. Finally, a significance test is ran on the data.

By default, the significance test is a two-sample Student’s t-test. Technically, the p-value from this test is the probability that a t-statistic larger than the one observed would’ve been observed if only chance were at play, but the walkthrough text says that is the probability that a mean difference larger than the one observed would’ve been observed if only chance were at play. That is, I use the t-test as an approximation to a permutation test. Switch on pedant mode if you want to run a permutation test.

## Parameters

`n`

: the number of data points per group`diff`

: The boost that participants in the intervention group receive.`sd`

: The standard deviation of the normal distribution from which the data were generated.`showdata`

: Do you want to output a dataframe containing the plotted data (TRUE) or not (FALSE, default)?`pedant`

: Do you want to run the significance test in pedant mode (TRUE; performs a permutation test) or not (FALSE, default; performs Student’s t-test)?