---
title: "Back to the Future: Human-friendly Scheffé Contrasts, or, the Art of Complex Multiple Comparisons... or... Everything Old Is New Again"
subtitle: "Gordon P Brooks & Nina Adjanin"
author: "brooksg@ohio.edu & adanin@nwmissouri.edu"
date: "Presented at AERA 2023 Chicago, 14-April-2023"
output:
  html_document:
    toc: true
    toc_level: 1
    number_sections: false
  pdf_document:
    toc: yes
  word_document:
    toc: no
classoption: portrait
editor_options:
  chunk_output_type: console
---

```{r KNIT_setup, include=FALSE, echo=FALSE, comment=NA}
knitr::opts_chunk$set(echo = TRUE)
```

# Program for "Barcikowski Human-Friendly" Contrasts

<!--
OPTIONS FOR YOU TO SET IN FIRST CODE CHUNK

* myAlpha governs how many Human-Friendly Contrasts are printed, defualt is all significant at alpha = .15 (You can change myAlpha here or later)

* onlyMax governs whether to give full results, including Brown-Forsythe adjusted Scheffe, only for the one Maximum Human-Friendly contrast (TRUE) or for all statistically significant at myAlpha, but at least 4 are always reported (you can change onlyMax here or later)

* printCons govern whether to print all Human-Friendly Contrasts tested (TRUE) or print none (you can change printCons here or later). Note that with more than 4 groups this output can get quite substantial...

* myExample allows you to try different data sets with the program... once you set YOUR OWN DATA to a data frame called myData it will override the examples
  * Stevens = IRS efficiency data (4 groups) from Stevens (1990) chapter problems
  * Snedecor = birth weight of pigs data (8 groups) was used by Hollingsworth (1978) and Williams (1979/1980)
  * Maxwell = hypertension data (4 groups) Maxwell, Delaney, & Kelley (2018) textbook
  * effective = generated data (3 groups) data from authors
  * ThreeGrps = generated data (3 groups) from authors
  * MC4G = generated data (4 groups) from authors
  * TAM = data (3 groups) collected by authors (0=Oc,1=LapY, 2=PhY)
-->

```{r SET_UP_OPTIONS, include=FALSE, echo=FALSE, comment=NA}
  myAlpha <- 0.15
  onlyMax <- TRUE
  printCons <- FALSE
  myExample <- "Stevens" #"Stevens" "Snedecor" "effective" "TAM" "Snedecor9" "Maxwell" "MC4G" "ThreeGrps"
```

## Citation

Brooks, G. P., & Adjanin, N. (2023, April). _Back to the Future: Human-friendly Scheffé Contrasts, or, the Art of Multiple Comparisons_. Paper presented at the 2023 annual meeting of the American Educational Research Association, Chicago, IL.

## Background

* The idea for "Human-Friendly Contrasts" in One-way ANOVA was originally conceived by Robert S. Barcikowski (2000, unpublished) based on Scheffe (1956) and Hollingsworth (1978). Barcikowski's original FORTRAN code was amended and adapted by Brooks & Adjanin.
* "Human-Friendly Contrasts" are based primarily on Helmert-type contrasts, providing "reasonable" comparisons among multiple groups
* For example, in an experiment with one control group and two treatment groups, a "reasonable" comparison might be "SOMETHING" versus "NOTHING". Such a comparison would require contrasts of the form:
  * Contrast Coefficients: Coef1 = 1.0, Coef2 = -0.50, Coef3 = -0.50
* Or maybe two trait/demographic groups (combined in a reasonable way, perhaps Associate and Full Tenured Professors) versus two other groups (combined reasonably, such as Assistant Professors and Instructors), a comparison might be "TENURED" versus "NOT TENURED". Such a comparison would require contrasts of the form:
  * Contrast Coefficients: Coef1 = 0.5, Coef2 = 0.50, Coef3 = -0.50, Coef4 = -0.50
* Maximum number of groups allowed by the program = 8

## qqBarth R package created by the authors

* https://people.ohio.edu/brooksg/qqBarth.tar.gz
* Maximum Scheffe Contrasts and Human-Friendly Contrasts
* Version: 1.4
* Description: Functions Identify the maximum Scheffe contrast and analyzes the contrasts for an applied researcher dataset. The code also identifies the slightly different normalized maximum posttest contrast described by Hollingsworth (1978). Critically, however, the coefficient weights from complex contrasts such as these are likely to be uninterpretable. Therefore, we have also implemented a method to identify maximum human-friendly contrasts that may be meaningful to and interpretable by a human researcher. We have named these human-friendly contrasts _Barcikowski contrasts_ in memory of Robert Barcikowski, the scholar who introduced the idea to the authors.

<!--
### Load Package

You can (and maybe should) continue to install the qqBarth package every time you run the program, at least until the package becomes available on CRAN. But until then, there may be frequent changes to the package. 
-->

```{r LOAD_qqBarth_PACKAGE, echo=FALSE, include=FALSE}
  pkg1 <- c("installr")
  ins1 <- pkg1[!(pkg1 %in% installed.packages()[,"Package"])]
  if(length(ins1)) install.packages(ins1, dependencies = TRUE)
  invisible(lapply(pkg1, library, character.only = TRUE))
#--
  pkg2 <- c("qqBarth")
  ins2 <- pkg2[!(pkg2 %in% installed.packages()[,"Package"])]
  installr::install.packages.zip ("https://people.ohio.edu/brooksg/qqBarth.tar.gz")
  invisible(lapply(pkg2, library, character.only = TRUE))
#--
  examples <- qqBarth::exampleDataFun()
```

### Data (import or create a data frame called *myData*)

* Put your data into a data frame called myData before running any more code
* Your grouping variable must be column 1 and your dependent variable must be column 2
* Replace myAlpha to some alpha value to set your desired alpha level to use for output of Human-Friendly Contrasts (you will always get at least 4 printed out)
* This code will show your first 6 cases, just to check

```{r SETUP_myDF, echo=FALSE}
  XCOL <- 1
  YCOL <- 2
  alpha2use <- myAlpha
  if (!exists("myData")) myData <- examples[[myExample]]
  myDF <- as.data.frame(myData)
  myDF <- data.frame(x=as.factor(myDF[,XCOL]), y=as.numeric(myDF[,YCOL]))
  myDF$x <- factor(as.numeric(myDF$x))
  head(myDF)
```


```{r RUN_FUNCTIONS_CREATE_OUTPUT, include=FALSE}
  OUT <- qqBarth::humanContrastsFun(myDF = myDF, XCOL = 1, YCOL = 2, ALPHA = c(.10, .05, .01), alpha2use = alpha2use)
  ifelse(OUT$gnum < 9, OK <- TRUE, OK <- FALSE)
```

# OUTPUT: Barcikowski's "Human-Friendly" Contrasts

```{r DATA_ERROR, include=TRUE, echo=FALSE, eval=!OK, include=!OK, comment=NA}
  knitr::asis_output("# DATA ERROR\\n")
  print("There is no OUTPUT because the data set contains more than 8 levels/groups")
  print(paste("There were ", OUT$gnum, " levels/groups identified for the x variable in these data"))
```


```{r TOTAL_SAMPLE_DESCRIPTIVES, include=TRUE, echo=FALSE, eval=OK, include=OK, comment=NA}
  knitr::asis_output("## Total Sample Descriptive Statistics\\n")
  qqBarth::totalSampleDescriptivesFun(OUT = OUT)
```


```{r BY_GROUP_DESCRIPTIVES, include=TRUE, echo=FALSE, include=OK, comment=NA}
  knitr::asis_output("## By-Group Descriptive Statistics\\n")
  qqBarth::byGroupDescriptivesFun(OUT = OUT)
```


```{r NORMALITY_OF_RESIDUALS, include=TRUE, echo=FALSE, include=OK, comment=NA}
  knitr::asis_output("## Test of Normality (of residuals)\\n")
  print(OUT$nort$ShapiroResid)
```

`r  if(OK) {"## Tests of Homoscedasticity \n Levene (mean)"}`

```{r LEVENE_MEAN_Homoscedasticity, include=TRUE, echo=FALSE, include=OK, comment=NA}
  print(OUT$sced$leveneMean1way)
```

`r  if(OK) {" Levene (median)"}`

```{r LEVENE_MEDIAN_Homoscedasticity, include=TRUE, echo=FALSE, include=OK, comment=NA}
  print(OUT$sced$leveneMedian1way)
```

`r  if(OK) {" Breusch-Pagan"}`

```{r BREUSCH_PAGAN_Homoscedasticity, include=TRUE, echo=FALSE, include=OK, comment=NA}
  print(OUT$sced$bptest1way)
```

`r  if(OK) {"## Omnibus (Overall) ANOVA \n Fisher F"}`

```{r FISHER_ANOVA_F, include=TRUE, echo=FALSE, include=OK, comment=NA}
  print(summary(OUT$omni$aov1way))
```

`r  if(OK) {" Robust Welch F"}`

```{r ROBUST_WELCH_F, include=TRUE, echo=FALSE, include=OK, comment=NA}
  print(OUT$omni$welch1way)
```

`r  if(OK) {" Robust Brown-Forsythe F"}`

```{r ROBUST_BROWN_FORSYTHE_F, include=TRUE, echo=FALSE, include=OK, comment=NA}
  cat("BROWN-FORSYTHE F", "\n\n", "F =", OUT$omni$bf1way[['statistic']], ", num df =", OUT$omni$bf1way[['parameter']][1],  ", den df =", OUT$omni$bf1way[['parameter']][2], ", p-value = ", OUT$omni$bf1way[['p.value']], '\n\n' )
```

<!--
If you want to see all contrasts tested, change printCons to TRUE at beginning of the program or the change *printCons* to TRUE in the code chunk below:
-->
`r  if(OK) {"# Barcikowski's Most Explanatory Human-Friendly Contrasts \n * The most explanatory contrasts significant at chosen alpha (at least 4 are shown with the MAXIMUM Human Contrast listed first) \n\n * The p values reported in the section below have already been adjusted using Scheffe adjustments, so compare the p value to your desired alpha level without further adjustment \n\n * A later section provides the Brown-Forsythe adjustment for unequal variances (for the Maximum contrast only by default, but can be changed to all those significant at chosen alpha)"}`

```{r SIG_OR_TOP4_HUMAN_FRIENDLY_CONTRASTS, include=TRUE, echo=FALSE, include=OK, comment=NA}
  cat(OUT$human$CONTOT, "total Human-Friendly Contrasts were tested using alpha = ", alpha2use, "\n\n")
  if (printCons) print(OUT$human$ALLCON, digits=2)
  qqBarth:: humanMaxFun(OUT = OUT, OK = OK)
```

`r  if(OK) {"# Maximum Contrasts (Barcikowski, Scheffe, Hollingsworth) \n * These are the Maximum contrasts for all methods for comparison purposes"}`

```{r MAXIMUM_CONTRASTS_ALL_METHODS, include=TRUE, echo=FALSE, include=OK, comment=NA}
  qqBarth::maximumContrastsFun(OUT = OUT, OK = OK)
```

`r  if(OK) {"## Tests of Scheffe-only contrasts without Brown-Forsythe adjustment \n * These analyses performed with DescTools ScheffeTest function with the maximum contrasts as input \n\n * The p values reported in sections below have already been adjusted using Scheffe adjustments, so compare the p value to your desired alpha level without further adjustment \n\n * The sections following this one provide the Brown-Forsythe adjustments for unequal variances for these Maximum Contrasts"}`

```{r SIG_MAXIMUM_CONTRASTS_ALL_METHODS, include=TRUE, echo=FALSE, include=OK, comment=NA}
  qqBarth::descToolsMaximumsFun(OUT = OUT, OK = OK)
```

<!--
Change _onlyMax = FALSE_ to obtain all Human-Friendly Contrasts significant at chosen alpha (or at least the 4 most explanatory contrasts)
-->
`r  if(OK) {"## Tests of Scheffe contrasts with Brown-Forsythe adjustment \n"}`
`r  if(OK) {"### BARCIKOWSKI HUMAN-FRIENDLY Maximum Contrast(s) \n * The sections below use calculations of statistics and critical values for statistical significance as opposed to the p values provided by the DescTools output above \n\n * One critically important part of the results sections below is the robust Brown-Forsythe Adjustment applied to Scheffe for unequal variances, something not available many places"}`

```{r HUMAN_FRIENDLY_MAXIMUM, include=TRUE, echo=FALSE, include=OK, comment=NA}
  qqBarth::top4Fun(OUT = OUT, onlyMax = onlyMax, OK = OK)
```

`r  if(OK) {"### HOLLINGSWORTH Maximum Contrast \n * Hollingsworth, H. (1978). The coefficients of the normalized maximum contrast as statistics for posttest ANOVA data interpretations. Journal of Experimental Education, 46(4), 4-6.\n\n * Hollingsworth, H. (1980/1981). Maximized posttest contrasts: A clarification. Journal of Experimental Education, 49(2), 92-93."}`

```{r HOLLINGSWORTH_MAXIMUM, include=TRUE, echo=FALSE, include=OK, comment=NA}
  qqBarth::hollingsworthMaxFun(OUT = OUT, OK = OK)
```

`r  if(OK) {"### SCHEFFE (SCALED) Maximum Contrast \n SCHEFFE Maximum Contrast & Maximum Contrast SCALED so positive & negative coefficient sets sum to 0"}`

```{r SCHEFFE_MAXIMUM, include=TRUE, echo=FALSE, include=OK, comment=NA}
  qqBarth::scheffeMaxFun(OUT = OUT, OK = OK)
```

# END
