WGA R Package & Tutorial
Our package builds on the multilevel package developed by Paul Bliese. However, our package differs from the multilevel in two important ways. First, our functions require a bit more information in order to compute the agreement statistics. Second, the burden of providing additional information is offset, we believe, with the generation of more comprehensive output, designed to be easily integrated into descriptions of data aggregation.
By generating a more comprehensive set of results, we hope researchers will use these results to enhance the clarity, transparency, and reproducibility of future multilevel research. Below we provide a brief tutorial on the use of our package. At this point, our package is hosted on GitHub, but we plan to make it available via CRAN, too.
Basic Functions Included with WGA: RWG, RWGJ, AWG, & AD
To illustrate the different functions, we walk the reader through a simple tutorial. The following code can be pasted into R or RStudio. Lines 1 and 2 install and load the “devtools” package into the library of active resources. Line 3 installs the “WGA” package.
- install.packages(“devtools”, repos = “http://cran.us.r-project.org”)
- library(devtools)
- devtools::install_github(“james-lebreton/WGA”, force=T)
- library(WGA)
Once the “WGA” package is downloaded from github, you may load it into your library of active resources.
Step 1 – Install Packages
install.packages("devtools", repos = "<http://cran.us.r-project.org>")
library(devtools)
devtools::install_github("james-lebreton/WGA", force=T, quiet=T)
library(WGA)Step 2 – Load Data
To illustrate our functions, we will use the lq2002 data that are included in the multilevel package. This package will be automatically loaded when using the WGA package. lq2002 contains data from 2042 soldiers nested in 49 companies in the U.S. Army. The following code loads the data and shows the first 10 variables for the first 10 soldiers.
data(lq2002, package = 'multilevel')
lq2002[1:10,1:15]## COMPID SUB LEAD01 LEAD02 LEAD03 LEAD04 LEAD05 LEAD06 LEAD07 LEAD08 LEAD09
## 1 2 1 2 2 2 3 2 2 5 5 5
## 2 2 2 4 4 2 4 4 4 4 4 4
## 3 2 3 4 4 2 4 2 4 4 4 4
## 4 2 4 1 1 1 1 1 1 4 2 3
## 5 2 5 1 1 1 1 3 2 1 1 1
## 6 2 6 2 3 2 4 2 4 3 4 2
## 7 2 7 3 2 2 1 2 2 2 2 3
## 8 2 8 3 3 2 4 2 2 2 4 2
## 9 2 9 3 4 3 2 3 3 3 1 3
## 10 2 10 4 3 3 4 3 4 3 4 3
## LEAD10 LEAD11 TSIG01 TSIG02
## 1 5 4 1 5
## 2 4 3 3 4
## 3 4 2 5 5
## 4 3 2 1 3
## 5 1 1 4 4
## 6 4 3 4 4
## 7 4 2 2 4
## 8 2 4 2 2
## 9 2 3 4 3
## 10 4 3 4 4Step 3a - Estimate Agreement on a Single Item using rWG, aWG, AD
Some users may wish to estimate agreement using a specific agreement statistic. Thus, our package includes separate functions for estimating within-unit agreement when a single item is used (i.e., agreement between soldiers nested in a common Army company on a specific item asking about leadership climate).
Each of these functions requires the user to provide information about:
- x, the item of interest,
- grpid, the clustering or grouping variable,
- scale, a vector containing the lowest and highest values of
the response scale (e.g., scale = c(1,5)),
- model, a user-supplied description of the multilevel data
aggregation measurement model (e.g., “Consensus”),
- reset, a logical argument that takes values of TRUE or
FALSE and indicates whether or not out-of-range values of rWG
should be reset to zero, and
- cutoff, a user-supplied cutoff for justifying data aggregation.
Below we present the code we used to estimate rWG values for the sixth item measuring solider perceptions of leadership (i.e., lq2002$LEAD06) which was measured on a scale ranging from 1 to 5 (i.e., scale = c(1,5). We described our model as a consensus model and indicated that we did not want out-of-range values to be automatically reset to zero. Wet specified a cutoff reflecting a moderate degree of agreement (rWG > 0.50). We will focus our attention on the estimates of rWG computed using a uniform null distribution (reflecting no response biases) and those computed using a slightly skewed null distribution (reflecting a slight leniency bias). The latter allows us to consider that some soldiers may be concerned about making negative statements about their commanding officers. We saved the results as a new object denoted fit1.
fit1<-RWG(x=lq2002$LEAD06, grpid=lq2002$COMPID, scale=c(1,5), model="Consensus", reset=F, cutoff=.51)
Step 4a - Examine Results
The output generated by each of these functions is a list containing different pieces of information (i.e., elements). We can use the names function to see the labels assigned to each element in the list, before examining each of the elements. In addition, the previous code automatically generates histograms plotting the distribution of within-group agreement statistics.
names(fit1)## [1] "rwg.descriptives" "rwg.over.cutoff" "rwg.percentiles"
## [4] "rwg.out.of.range" "rwg.error.variances" "rwg.results"
## [7] "rwg.plots" "rwg.p"We now examine each element in fit1. The first element provides a descriptive statistics summary of the statistics used to help justify data aggregation. From fit1, we see, most groups (on average) had a relatively modest level of agreement on the LEAD06 item. Using a uniform null distribution, the mean rWG value was .51. Not surprisingly, this mean value decreased when examining agreement in the presence of potential response biases. In addition, as we shrink the size of the null error variance used to estimate rWG (i.e., by using estimates that account for various response biases), we are more likely to obtain out-of-range estimates of rWG (i.e., rWG < 0.00).
fit1$rwg.descriptives## vars n mean sd median trimmed mad min max range skew
## grp.size 1 49 41.67 27.57 30.00 39.73 25.20 10.00 99.00 89.00 0.52
## num.items 2 49 1.00 0.00 1.00 1.00 0.00 1.00 1.00 0.00 NaN
## item.var 3 49 0.99 0.31 0.93 0.97 0.33 0.50 1.62 1.12 0.30
## rwg.un 4 49 0.51 0.16 0.53 0.51 0.16 0.19 0.75 0.56 -0.29
## rwg.ss 5 49 0.26 0.23 0.31 0.27 0.25 -0.21 0.63 0.84 -0.30
## rwg.ms 6 49 -0.10 0.34 -0.03 -0.08 0.37 -0.80 0.44 1.24 -0.30
## rwg.hs 7 49 -1.24 0.70 -1.11 -1.21 0.74 -2.68 -0.14 2.54 -0.30
## rwg.tri 8 49 0.25 0.23 0.30 0.26 0.25 -0.23 0.62 0.85 -0.31
## rwg.nor 9 49 0.05 0.30 0.11 0.06 0.33 -0.56 0.52 1.08 -0.30
## kurtosis se
## grp.size -1.17 3.94
## num.items NaN 0.00
## item.var -0.85 0.04
## rwg.un -0.87 0.02
## rwg.ss -0.86 0.03
## rwg.ms -0.85 0.05
## rwg.hs -0.85 0.10
## rwg.tri -0.84 0.03
## rwg.nor -0.85 0.04The next element in fit1 summarizes the proportion of groups with rWG values exceeding the user-specified cutoff (rWG > 0.51 in our example). Given that this analysis is for a single item (LEAD06), it is not surprising to see only 59% of the groups had rwg.un values greater than our cutoff.
fit1$rwg.over.cutoff## $rwg.un.cutoff
## [1] 0.59
##
## $rwg.ss.cutoff
## [1] 0.16
##
## $rwg.ms.cutoff
## [1] 0
##
## $rwg.hs.cutoff
## [1] 0
##
## $rwg.tri.cutoff
## [1] 0.14
##
## $rwg.nor.cutoff
## [1] 0.04The next element in fit1 summaries the distribution of rWG values, broken into percentiles. From this distribution, we get a better picture of how agreement varies across the 49 groups. Specifically, examining the distribution of rwg.un estimates, we see that few groups (~10%) had values of rWG that were indicative of “strong agreement” (i.e., > 0.70 using the heuristics recommended by LeBreton and Senter (2008)). Specifically, the 90th percentile for rwg.un was only 0.72.
fit1$rwg.percentiles## $rwg.un
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## 0.190 0.314 0.356 0.418 0.488 0.530 0.558 0.600 0.628 0.722 0.750
##
## $rwg.ss
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## -0.210 -0.020 0.036 0.122 0.232 0.310 0.340 0.400 0.442 0.584 0.630
##
## $rwg.ms
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## -0.800 -0.524 -0.428 -0.304 -0.152 -0.030 0.018 0.100 0.172 0.384 0.440
##
## $rwg.hs
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## -2.680 -2.118 -1.926 -1.664 -1.352 -1.110 -1.004 -0.840 -0.702 -0.262 -0.140
##
## $rwg.tri
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## -0.230 -0.040 0.026 0.112 0.214 0.300 0.330 0.390 0.432 0.582 0.620
##
## $rwg.nor
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## -0.560 -0.322 -0.238 -0.128 0.006 0.110 0.148 0.220 0.282 0.464 0.520The next element summarizes how many of the groups had out-of-range rWG values (i.e., rWG < 0.00) and clearly states whether the researcher requested that out-of-range values be reset to zero or retained in the analysis. Not surprisingly, when rWG was estimated using heavily skewed null distributions or those reflecting a strong central tendency bias, more groups had negative estimates of agreement. The most likely explanation for these negative values that these distributions are not appropriate for this particular analysis. As noted earlier, we are focusing our attention on the estimates of rWG computed using the uniform null and the slightly skewed null. For both of those distributions, none of the estimates of rWG were out-of-range.
fit1$rwg.out.of.range## num.oor.un num.oor.ss num.oor.ms num.oor.hs num.oor.tri num.oor.nor
## 1 0 0 6 25 49 6
## reset.to.zero
## 1 NoThe next element clarifies the number of scale points on our item and lists the specific point-estimates for the null error variances (based on different null distributions) that were used to compute estimates of rWG. The RWG function computes agreement using uniform (uni), slightly skewed (ss), moderately skewed (ms), highly skewed (hs), triangular (tri), and normal (nor) null distributions.
fit1$rwg.error.variances## scale.points uni ss ms hs tri nor
## 1 5 2 1.34 0.9 0.44 1.32 1.04The next element is a data frame containing information about group sizes and the rWG estimates for each group. This information may be easily merged with the original data.
fit1$rwg.results## grp.name grp.size aggr.model num.items item.var rwg.un rwg.ss rwg.ms rwg.hs
## 2 2 24 Consensus 1 1.28 0.36 0.04 -0.42 -1.91
## 3 3 37 Consensus 1 0.88 0.56 0.34 0.02 -1.00
## 4 4 45 Consensus 1 1.27 0.36 0.05 -0.41 -1.89
## 5 5 58 Consensus 1 1.19 0.41 0.11 -0.32 -1.70
## 6 6 12 Consensus 1 1.30 0.35 0.03 -0.44 -1.95
## 7 7 15 Consensus 1 0.50 0.75 0.63 0.44 -0.14
## 9 9 13 Consensus 1 0.56 0.72 0.58 0.38 -0.27
## 10 10 29 Consensus 1 0.96 0.52 0.28 -0.07 -1.18
## 13 13 99 Consensus 1 1.12 0.44 0.16 -0.24 -1.55
## 14 14 10 Consensus 1 0.84 0.58 0.37 0.07 -0.91
## 15 15 89 Consensus 1 0.76 0.62 0.43 0.16 -0.73
## 16 16 23 Consensus 1 1.52 0.24 -0.13 -0.69 -2.45
## 17 17 10 Consensus 1 1.07 0.46 0.20 -0.19 -1.43
## 18 18 94 Consensus 1 1.42 0.29 -0.06 -0.58 -2.23
## 19 19 13 Consensus 1 0.77 0.62 0.43 0.14 -0.75
## 20 20 68 Consensus 1 1.62 0.19 -0.21 -0.80 -2.68
## 21 21 53 Consensus 1 0.89 0.55 0.34 0.01 -1.02
## 22 22 25 Consensus 1 0.93 0.53 0.31 -0.03 -1.11
## 23 23 54 Consensus 1 0.82 0.59 0.39 0.09 -0.86
## 24 24 37 Consensus 1 1.36 0.32 -0.01 -0.51 -2.09
## 25 25 41 Consensus 1 0.97 0.52 0.28 -0.08 -1.20
## 26 26 21 Consensus 1 1.31 0.34 0.02 -0.46 -1.98
## 27 27 73 Consensus 1 0.96 0.52 0.28 -0.07 -1.18
## 28 28 68 Consensus 1 0.80 0.60 0.40 0.11 -0.82
## 29 29 85 Consensus 1 0.81 0.60 0.40 0.10 -0.84
## 30 30 29 Consensus 1 0.51 0.74 0.62 0.43 -0.16
## 31 31 19 Consensus 1 0.81 0.60 0.40 0.10 -0.84
## 32 32 68 Consensus 1 1.30 0.35 0.03 -0.44 -1.95
## 33 33 77 Consensus 1 1.15 0.43 0.14 -0.28 -1.61
## 34 34 78 Consensus 1 1.05 0.48 0.22 -0.17 -1.39
## 35 35 58 Consensus 1 0.73 0.64 0.46 0.19 -0.66
## 37 37 10 Consensus 1 0.54 0.73 0.60 0.40 -0.23
## 38 38 30 Consensus 1 0.79 0.60 0.41 0.12 -0.80
## 41 41 78 Consensus 1 0.66 0.67 0.51 0.27 -0.50
## 42 42 54 Consensus 1 0.93 0.53 0.31 -0.03 -1.11
## 43 43 74 Consensus 1 1.07 0.46 0.20 -0.19 -1.43
## 44 44 63 Consensus 1 0.90 0.55 0.33 0.00 -1.05
## 45 45 30 Consensus 1 0.87 0.56 0.35 0.03 -0.98
## 46 46 90 Consensus 1 0.68 0.66 0.49 0.24 -0.55
## 47 47 14 Consensus 1 0.90 0.55 0.33 0.00 -1.05
## 48 48 13 Consensus 1 1.26 0.37 0.06 -0.40 -1.86
## 49 49 18 Consensus 1 1.56 0.22 -0.16 -0.73 -2.55
## 50 50 23 Consensus 1 0.53 0.74 0.60 0.41 -0.20
## 52 52 39 Consensus 1 1.31 0.34 0.02 -0.46 -1.98
## 54 54 19 Consensus 1 0.60 0.70 0.55 0.33 -0.36
## 55 55 11 Consensus 1 1.62 0.19 -0.21 -0.80 -2.68
## 56 56 18 Consensus 1 1.23 0.38 0.08 -0.37 -1.80
## 57 57 13 Consensus 1 0.90 0.55 0.33 0.00 -1.05
## 58 58 20 Consensus 1 0.53 0.74 0.60 0.41 -0.20
## rwg.tri rwg.nor
## 2 0.03 -0.23
## 3 0.33 0.15
## 4 0.04 -0.22
## 5 0.10 -0.14
## 6 0.02 -0.25
## 7 0.62 0.52
## 9 0.58 0.46
## 10 0.27 0.08
## 13 0.15 -0.08
## 14 0.36 0.19
## 15 0.42 0.27
## 16 -0.15 -0.46
## 17 0.19 -0.03
## 18 -0.08 -0.37
## 19 0.42 0.26
## 20 -0.23 -0.56
## 21 0.33 0.14
## 22 0.30 0.11
## 23 0.38 0.21
## 24 -0.03 -0.31
## 25 0.27 0.07
## 26 0.01 -0.26
## 27 0.27 0.08
## 28 0.39 0.23
## 29 0.39 0.22
## 30 0.61 0.51
## 31 0.39 0.22
## 32 0.02 -0.25
## 33 0.13 -0.11
## 34 0.20 -0.01
## 35 0.45 0.30
## 37 0.59 0.48
## 38 0.40 0.24
## 41 0.50 0.37
## 42 0.30 0.11
## 43 0.19 -0.03
## 44 0.32 0.13
## 45 0.34 0.16
## 46 0.48 0.35
## 47 0.32 0.13
## 48 0.05 -0.21
## 49 -0.18 -0.50
## 50 0.60 0.49
## 52 0.01 -0.26
## 54 0.55 0.42
## 55 -0.23 -0.56
## 56 0.07 -0.18
## 57 0.32 0.13
## 58 0.60 0.49lq2002 <- merge(x = lq2002, y = fit1$rwg.results, by.x = "COMPID", by.y = "grp.name")
names(lq2002)## [1] "COMPID" "SUB" "LEAD01" "LEAD02" "LEAD03"
## [6] "LEAD04" "LEAD05" "LEAD06" "LEAD07" "LEAD08"
## [11] "LEAD09" "LEAD10" "LEAD11" "TSIG01" "TSIG02"
## [16] "TSIG03" "HOSTIL01" "HOSTIL02" "HOSTIL03" "HOSTIL04"
## [21] "HOSTIL05" "LEAD" "TSIG" "HOSTILE" "GLEAD"
## [26] "GTSIG" "GHOSTILE" "grp.size" "aggr.model" "num.items"
## [31] "item.var" "rwg.un" "rwg.ss" "rwg.ms" "rwg.hs"
## [36] "rwg.tri" "rwg.nor"The next element, rwg.plots is empty. However, when the RWG function was executed, it automatically generated a set of histograms that could be used to visually depict the distribution of agreement, conditional on different response biases (e.g., slightly skewed error null distribution is used to account for a slight leniency or severity bias; see output immediately following the code that generated fit1).
fit1$rwg.plots## NULLThe final element, rwg.p, contains the overall or omnibus estimates of rWG based on using a pooled, within-groups estimate of variance. The advantage of rWGp is that is furnishes a single estimate of agreement for an entire data set rather than separate estimates of agreement for each group. Thus, rWGp allows researchers to get a sense of overall agreement based on the pooling of within-group variances. Based on the results, we see that using a uniform null, our estimate of rwgp.un = 0.50. This this value drops considerably when using a slightly skewed distribution to rwgp.ss = 0.25.
Overall, our results indicate that, on average, there is relative week agreement within groups. Examining the rwg.descriptives and rwg.results indicates that there were some groups with moderate to strong agreement, but many groups with weak agreement. Thus, we would likely conclude that there is not sufficient agreement to warrant aggregating LEAD06 to the group level and treating it as an indicator for a group-level construct.
fit1$rwg.p## $rwgp.un
## [1] 0.5
##
## $rwgp.ss
## [1] 0.25
##
## $rwgp.ms
## [1] -0.11
##
## $rwgp.hs
## [1] -1.27
##
## $rwgp.tri
## [1] 0.24
##
## $rwgp.nor
## [1] 0.04Step 3b - Estimate Agreement on a Multi-Item Scale using rWG(J)
Next we demonstrate how to estimate within-group agreement for multiple item scales. A separate function is called to estimate rWG(J). The following code is used to estimate within-group agreement for the 11 items measuring soldier perceptions of leadership climate. These items are denoted “LEAD01” to “LEAD11” and occupy columns 3 to 13 in the lq2002 data frame. We have also changed the reset argument to request that any out-of-range estimates of rWG(J) be reset to zero.
fit2<-RWGJ(x=lq2002[,c(3:13)], grpid=lq2002$COMPID, scale=c(1,5), model="Consensus", reset=T, cutoff=0.70)
Step 4b - Examine Results
We can examine individual elements in the results list or we can simply review all of the results for a particular analysis. For example, to review the results for rWG(J), we would just type fit2 into the console and hit ‘enter’.
fit2## $rwgj.descriptives
## vars n mean sd median trimmed mad min max range skew
## grp.size 1 49 41.67 27.57 30.00 39.73 25.20 10.00 99.00 89.00 0.52
## num.items 2 49 11.00 0.00 11.00 11.00 0.00 11.00 11.00 0.00 NaN
## mean.item.var 3 49 0.14 0.33 0.00 0.08 0.00 0.00 1.00 1.00 1.79
## rwgj.un 4 49 0.87 0.05 0.87 0.87 0.06 0.77 0.94 0.17 -0.23
## rwgj.ss 5 49 0.41 0.34 0.46 0.41 0.49 0.00 0.88 0.88 -0.10
## rwgj.ms 6 49 0.04 0.13 0.00 0.00 0.00 0.00 0.57 0.57 3.21
## rwgj.hs 7 49 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 NaN
## rwgj.tri 8 49 0.39 0.34 0.41 0.38 0.61 0.00 0.88 0.88 0.01
## rwgj.nor 9 49 0.10 0.24 0.00 0.05 0.00 0.00 0.76 0.76 1.93
## kurtosis se
## grp.size -1.17 3.94
## num.items NaN 0.00
## mean.item.var 1.27 0.05
## rwgj.un -1.02 0.01
## rwgj.ss -1.69 0.05
## rwgj.ms 9.38 0.02
## rwgj.hs NaN 0.00
## rwgj.tri -1.71 0.05
## rwgj.nor 1.94 0.03
##
## $rwgj.over.cutoff
## $rwgj.over.cutoff$rwgj.un.cutoff
## [1] 1
##
## $rwgj.over.cutoff$rwgj.ss.cutoff
## [1] 0.29
##
## $rwgj.over.cutoff$rwgj.ms.cutoff
## [1] 0
##
## $rwgj.over.cutoff$rwgj.hs.cutoff
## [1] 0
##
## $rwgj.over.cutoff$rwgj.tri.cutoff
## [1] 0.24
##
## $rwgj.over.cutoff$rwgj.nor.cutoff
## [1] 0.06
##
##
## $rwgj.percentiles
## $rwgj.percentiles$rwgj.un
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## 0.77 0.80 0.83 0.84 0.86 0.87 0.89 0.90 0.91 0.93 0.94
##
## $rwgj.percentiles$rwgj.ss
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## 0.000 0.000 0.000 0.000 0.336 0.460 0.618 0.690 0.744 0.842 0.880
##
## $rwgj.percentiles$rwgj.ms
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.036 0.570
##
## $rwgj.percentiles$rwgj.hs
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## 0 0 0 0 0 0 0 0 0 0 0
##
## $rwgj.percentiles$rwgj.tri
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## 0.000 0.000 0.000 0.000 0.246 0.410 0.596 0.670 0.724 0.834 0.880
##
## $rwgj.percentiles$rwgj.nor
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.62 0.76
##
##
## $rwgj.out.of.bounds
## num.oor.un num.oor.ss num.oor.ms num.oor.hs num.oor.tri num.oor.nor
## 1 41 0 16 44 49 18
## reset.to.zero
## 1 Yes
##
## $rwgj.error.variances
## scale.points uni ss ms hs tri nor
## 1 5 2 1.34 0.9 0.44 1.32 1.04
##
## $rwgj.results
## grp.name grp.size aggr.model num.items mean.item.var rwgj.un rwgj.ss rwgj.ms
## 2 2 24 Consensus 11 0.00 0.81 0.00 0.00
## 3 3 37 Consensus 11 0.00 0.90 0.68 0.00
## 4 4 45 Consensus 11 0.00 0.86 0.33 0.00
## 5 5 58 Consensus 11 0.00 0.86 0.38 0.00
## 6 6 12 Consensus 11 0.00 0.89 0.62 0.00
## 7 7 15 Consensus 11 0.88 0.93 0.85 0.18
## 9 9 13 Consensus 11 0.00 0.89 0.65 0.00
## 10 10 29 Consensus 11 0.00 0.85 0.13 0.00
## 13 13 99 Consensus 11 0.00 0.88 0.58 0.00
## 14 14 10 Consensus 11 0.00 0.83 0.00 0.00
## 15 15 89 Consensus 11 0.00 0.87 0.49 0.00
## 16 16 23 Consensus 11 0.00 0.82 0.00 0.00
## 17 17 10 Consensus 11 0.81 0.94 0.88 0.55
## 18 18 94 Consensus 11 0.00 0.80 0.00 0.00
## 19 19 13 Consensus 11 0.00 0.90 0.73 0.00
## 20 20 68 Consensus 11 0.00 0.77 0.00 0.00
## 21 21 53 Consensus 11 0.00 0.85 0.20 0.00
## 22 22 25 Consensus 11 0.00 0.87 0.46 0.00
## 23 23 54 Consensus 11 0.00 0.89 0.66 0.00
## 24 24 37 Consensus 11 0.00 0.78 0.00 0.00
## 25 25 41 Consensus 11 0.00 0.91 0.76 0.00
## 26 26 21 Consensus 11 0.00 0.87 0.40 0.00
## 27 27 73 Consensus 11 0.00 0.83 0.00 0.00
## 28 28 68 Consensus 11 0.00 0.91 0.75 0.00
## 29 29 85 Consensus 11 0.00 0.84 0.00 0.00
## 30 30 29 Consensus 11 0.00 0.91 0.74 0.00
## 31 31 19 Consensus 11 1.00 0.92 0.79 0.00
## 32 32 68 Consensus 11 0.00 0.81 0.00 0.00
## 33 33 77 Consensus 11 0.00 0.83 0.00 0.00
## 34 34 78 Consensus 11 0.00 0.84 0.01 0.00
## 35 35 58 Consensus 11 0.00 0.90 0.71 0.00
## 37 37 10 Consensus 11 0.91 0.93 0.84 0.00
## 38 38 30 Consensus 11 0.87 0.93 0.86 0.27
## 41 41 78 Consensus 11 0.00 0.90 0.69 0.00
## 42 42 54 Consensus 11 0.00 0.84 0.00 0.00
## 43 43 74 Consensus 11 0.00 0.87 0.46 0.00
## 44 44 63 Consensus 11 0.00 0.80 0.00 0.00
## 45 45 30 Consensus 11 0.00 0.90 0.69 0.00
## 46 46 90 Consensus 11 0.00 0.90 0.70 0.00
## 47 47 14 Consensus 11 0.00 0.89 0.61 0.00
## 48 48 13 Consensus 11 0.00 0.88 0.54 0.00
## 49 49 18 Consensus 11 0.00 0.83 0.00 0.00
## 50 50 23 Consensus 11 0.80 0.94 0.88 0.57
## 52 52 39 Consensus 11 0.00 0.79 0.00 0.00
## 54 54 19 Consensus 11 0.93 0.93 0.83 0.00
## 55 55 11 Consensus 11 0.00 0.86 0.36 0.00
## 56 56 18 Consensus 11 0.00 0.80 0.00 0.00
## 57 57 13 Consensus 11 0.00 0.83 0.00 0.00
## 58 58 20 Consensus 11 0.86 0.94 0.86 0.35
## rwgj.hs rwgj.tri rwgj.nor
## 2 0 0.00 0.00
## 3 0 0.66 0.00
## 4 0 0.24 0.00
## 5 0 0.30 0.00
## 6 0 0.60 0.00
## 7 0 0.85 0.66
## 9 0 0.62 0.00
## 10 0 0.00 0.00
## 13 0 0.55 0.00
## 14 0 0.00 0.00
## 15 0 0.43 0.00
## 16 0 0.00 0.00
## 17 0 0.87 0.76
## 18 0 0.00 0.00
## 19 0 0.71 0.00
## 20 0 0.00 0.00
## 21 0 0.07 0.00
## 22 0 0.41 0.00
## 23 0 0.64 0.00
## 24 0 0.00 0.00
## 25 0 0.74 0.00
## 26 0 0.33 0.00
## 27 0 0.00 0.00
## 28 0 0.73 0.00
## 29 0 0.00 0.00
## 30 0 0.72 0.00
## 31 0 0.78 0.30
## 32 0 0.00 0.00
## 33 0 0.00 0.00
## 34 0 0.00 0.00
## 35 0 0.69 0.00
## 37 0 0.83 0.61
## 38 0 0.85 0.68
## 41 0 0.67 0.00
## 42 0 0.00 0.00
## 43 0 0.40 0.00
## 44 0 0.00 0.00
## 45 0 0.67 0.00
## 46 0 0.69 0.00
## 47 0 0.58 0.00
## 48 0 0.50 0.00
## 49 0 0.00 0.00
## 50 0 0.88 0.76
## 52 0 0.00 0.00
## 54 0 0.82 0.56
## 55 0 0.27 0.00
## 56 0 0.00 0.00
## 57 0 0.00 0.00
## 58 0 0.86 0.70
##
## $rwgj.plots
## NULLWGA: An Integrated Function for Estimating Within-Group Agreement
Although some users may prefer to estimate agreement using the separate functions described in the previous section, we have also included the WGA function, which is a wrapper function that combines the previous analyses into a single set of output. The WGA function also computes additional estimates of agreement (e.g., AD, aWG) which may be of interest to some users.
Below is the code for estimating agreement on the LEAD06 item using the WGA function.
fit3 <- WGA(x=lq2002[,c("LEAD01")], grpid=lq2002$COMPID, scale=c(1,5),
model="Consensus", reset=T, cutoff=0.70)
fit3## $data.aggreation.model
## [1] "Consensus"
##
## $wga.descriptives
## vars n mean sd median trimmed mad min max range skew
## grp.size 1 49 41.67 27.57 30.00 39.73 25.20 10.00 99.00 89.00 0.52
## num.items 2 49 1.00 0.00 1.00 1.00 0.00 1.00 1.00 0.00 NaN
## item.var 3 49 1.15 0.30 1.16 1.16 0.25 0.41 1.76 1.35 -0.35
## rwg.un 4 49 0.42 0.15 0.42 0.42 0.12 0.12 0.80 0.68 0.37
## rwg.ss 5 49 0.17 0.18 0.13 0.15 0.19 0.00 0.69 0.69 1.10
## rwg.ms 6 49 0.04 0.12 0.00 0.01 0.00 0.00 0.54 0.54 2.86
## rwg.hs 7 49 0.00 0.01 0.00 0.00 0.00 0.00 0.07 0.07 6.58
## rwg.tri 8 49 0.16 0.18 0.12 0.14 0.18 0.00 0.69 0.69 1.18
## rwg.nor 9 49 0.07 0.15 0.00 0.04 0.00 0.00 0.61 0.61 2.27
## awg 10 49 0.43 0.14 0.43 0.42 0.13 0.13 0.76 0.63 0.33
## AD.mean 11 49 0.86 0.16 0.88 0.87 0.13 0.30 1.10 0.80 -1.04
## AD.median 12 49 0.81 0.18 0.82 0.82 0.15 0.23 1.07 0.84 -0.95
## kurtosis se
## grp.size -1.17 3.94
## num.items NaN 0.00
## item.var -0.15 0.04
## rwg.un -0.10 0.02
## rwg.ss 0.45 0.03
## rwg.ms 7.42 0.02
## rwg.hs 42.12 0.00
## rwg.tri 0.63 0.03
## rwg.nor 4.27 0.02
## awg -0.20 0.02
## AD.mean 1.72 0.02
## AD.median 1.09 0.03
##
## $rwg.over.cutoff
## $rwg.over.cutoff$rwg.un.cutoff
## [1] 0.06
##
## $rwg.over.cutoff$rwg.ss.cutoff
## [1] 0
##
## $rwg.over.cutoff$rwg.ms.cutoff
## [1] 0
##
## $rwg.over.cutoff$rwg.hs.cutoff
## [1] 0
##
## $rwg.over.cutoff$rwg.tri.cutoff
## [1] 0
##
## $rwg.over.cutoff$rwg.nor.cutoff
## [1] 0
##
##
## $rwg.out.of.bounds
## NULL
##
## $rwg.error.variances
## scale.points uni ss ms hs tri nor
## 1 5 2 1.34 0.9 0.44 1.32 1.04
##
## $wga.percentiles
## $wga.percentiles$rwg.un
## NULL
##
## $wga.percentiles$awg.un
## 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
## 0.130 0.248 0.336 0.350 0.372 0.430 0.450 0.490 0.544 0.612 0.760
##
##
## $wga.results
## grp.name grp.size aggr.model num.items item.var rwg.un rwg.ss rwg.ms rwg.hs
## 2 2 24 Consensus 1 1.26 0.37 0.06 0.00 0.00
## 3 3 37 Consensus 1 1.61 0.19 0.00 0.00 0.00
## 4 4 45 Consensus 1 1.12 0.44 0.16 0.00 0.00
## 5 5 58 Consensus 1 0.99 0.50 0.26 0.00 0.00
## 6 6 12 Consensus 1 0.70 0.65 0.48 0.22 0.00
## 7 7 15 Consensus 1 1.12 0.44 0.16 0.00 0.00
## 9 9 13 Consensus 1 1.40 0.30 0.00 0.00 0.00
## 10 10 29 Consensus 1 1.19 0.41 0.11 0.00 0.00
## 13 13 99 Consensus 1 0.99 0.50 0.26 0.00 0.00
## 14 14 10 Consensus 1 1.16 0.42 0.13 0.00 0.00
## 15 15 89 Consensus 1 1.16 0.42 0.13 0.00 0.00
## 16 16 23 Consensus 1 1.51 0.24 0.00 0.00 0.00
## 17 17 10 Consensus 1 0.49 0.76 0.63 0.46 0.00
## 18 18 94 Consensus 1 1.09 0.45 0.19 0.00 0.00
## 19 19 13 Consensus 1 1.40 0.30 0.00 0.00 0.00
## 20 20 68 Consensus 1 1.76 0.12 0.00 0.00 0.00
## 21 21 53 Consensus 1 1.27 0.36 0.05 0.00 0.00
## 22 22 25 Consensus 1 1.39 0.31 0.00 0.00 0.00
## 23 23 54 Consensus 1 0.95 0.52 0.29 0.00 0.00
## 24 24 37 Consensus 1 1.64 0.18 0.00 0.00 0.00
## 25 25 41 Consensus 1 1.25 0.38 0.07 0.00 0.00
## 26 26 21 Consensus 1 1.11 0.44 0.17 0.00 0.00
## 27 27 73 Consensus 1 1.51 0.24 0.00 0.00 0.00
## 28 28 68 Consensus 1 1.13 0.44 0.16 0.00 0.00
## 29 29 85 Consensus 1 1.06 0.47 0.21 0.00 0.00
## 30 30 29 Consensus 1 1.17 0.42 0.13 0.00 0.00
## 31 31 19 Consensus 1 0.78 0.61 0.42 0.13 0.00
## 32 32 68 Consensus 1 1.25 0.38 0.07 0.00 0.00
## 33 33 77 Consensus 1 1.27 0.36 0.05 0.00 0.00
## 34 34 78 Consensus 1 1.62 0.19 0.00 0.00 0.00
## 35 35 58 Consensus 1 1.03 0.48 0.23 0.00 0.00
## 37 37 10 Consensus 1 0.99 0.50 0.26 0.00 0.00
## 38 38 30 Consensus 1 1.22 0.39 0.09 0.00 0.00
## 41 41 78 Consensus 1 1.28 0.36 0.04 0.00 0.00
## 42 42 54 Consensus 1 1.51 0.24 0.00 0.00 0.00
## 43 43 74 Consensus 1 0.88 0.56 0.34 0.02 0.00
## 44 44 63 Consensus 1 1.32 0.34 0.01 0.00 0.00
## 45 45 30 Consensus 1 0.95 0.52 0.29 0.00 0.00
## 46 46 90 Consensus 1 1.02 0.49 0.24 0.00 0.00
## 47 47 14 Consensus 1 0.57 0.72 0.57 0.37 0.00
## 48 48 13 Consensus 1 0.41 0.80 0.69 0.54 0.07
## 49 49 18 Consensus 1 1.08 0.46 0.19 0.00 0.00
## 50 50 23 Consensus 1 0.68 0.66 0.49 0.24 0.00
## 52 52 39 Consensus 1 1.33 0.33 0.01 0.00 0.00
## 54 54 19 Consensus 1 0.78 0.61 0.42 0.13 0.00
## 55 55 11 Consensus 1 1.36 0.32 0.00 0.00 0.00
## 56 56 18 Consensus 1 1.56 0.22 0.00 0.00 0.00
## 57 57 13 Consensus 1 1.26 0.37 0.06 0.00 0.00
## 58 58 20 Consensus 1 0.94 0.53 0.30 0.00 0.00
## rwg.tri rwg.nor awg AD.mean AD.median
## 2 0.05 0.00 0.40 0.97 0.96
## 3 0.00 0.00 0.22 1.04 1.03
## 4 0.15 0.00 0.45 0.81 0.76
## 5 0.25 0.05 0.51 0.71 0.67
## 6 0.47 0.33 0.61 0.58 0.50
## 7 0.15 0.00 0.45 0.84 0.67
## 9 0.00 0.00 0.34 1.05 1.00
## 10 0.10 0.00 0.33 0.82 0.72
## 13 0.25 0.05 0.50 0.81 0.80
## 14 0.12 0.00 0.46 0.88 0.80
## 15 0.12 0.00 0.43 0.86 0.85
## 16 0.00 0.00 0.26 1.03 0.96
## 17 0.63 0.53 0.76 0.56 0.40
## 18 0.17 0.00 0.44 0.86 0.77
## 19 0.00 0.00 0.34 0.95 0.92
## 20 0.00 0.00 0.13 1.10 1.07
## 21 0.04 0.00 0.38 0.95 0.94
## 22 0.00 0.00 0.33 0.97 0.96
## 23 0.28 0.09 0.52 0.82 0.80
## 24 0.00 0.00 0.20 1.06 1.05
## 25 0.05 0.00 0.34 0.95 0.88
## 26 0.16 0.00 0.46 0.91 0.81
## 27 0.00 0.00 0.24 1.05 0.96
## 28 0.14 0.00 0.44 0.84 0.82
## 29 0.20 0.00 0.46 0.87 0.82
## 30 0.11 0.00 0.37 0.90 0.86
## 31 0.41 0.25 0.58 0.71 0.63
## 32 0.05 0.00 0.38 0.91 0.87
## 33 0.04 0.00 0.37 0.95 0.92
## 34 0.00 0.00 0.20 1.07 1.06
## 35 0.22 0.01 0.49 0.84 0.79
## 37 0.25 0.05 0.55 0.74 0.70
## 38 0.08 0.00 0.36 0.89 0.80
## 41 0.03 0.00 0.34 0.95 0.92
## 42 0.00 0.00 0.25 1.03 1.00
## 43 0.33 0.15 0.56 0.77 0.72
## 44 0.00 0.00 0.35 0.90 0.89
## 45 0.28 0.09 0.54 0.77 0.73
## 46 0.23 0.02 0.49 0.81 0.78
## 47 0.57 0.45 0.71 0.63 0.57
## 48 0.69 0.61 0.76 0.30 0.23
## 49 0.18 0.00 0.44 0.88 0.83
## 50 0.48 0.35 0.63 0.62 0.48
## 52 0.00 0.00 0.35 0.97 0.92
## 54 0.41 0.25 0.62 0.71 0.63
## 55 0.00 0.00 0.38 0.81 0.73
## 56 0.00 0.00 0.26 1.07 1.06
## 57 0.05 0.00 0.36 0.97 0.92
## 58 0.29 0.10 0.55 0.73 0.70
##
## $wga.plots
## $wga.plots$rwg.plots
## NULL
##
## $wga.plots$awg.plots
## NULLAggregating the Data
If the results of the within-group agreement analyses justify aggregating lower-level scores to a higher level, researchers may use a new function called AGGREGATE.DATA. This function expands the aggregate function that is part of the stats package. The typical use of the aggregate function would require a researcher to use the function to create a new data set containing the aggregated data, then rename variables, and finally merge the original data frame (lower-level) with the new (aggregated) data frame. Our new function integrates these separate steps. We illustrate this function using the scale scores on the 11-item leadership climate questionnaire, denoted LEAD in the lq2002 data frame. Options for the aggregation statistic include: mean, median, var, sd, min, or max.
The AGGREGATE.DATA function creates two new data frames and adds them to the global environment.
df.aggr contains the clustering variable (e.g., Army company) and the results of the aggregation algorithm applied to the focal variable (e.g., mean scores on the leadership climate scale).
df.combined is simply the original data frame merged with df.aggr. As part of this merge process, the aggregated scores (e.g., Army company leadership climate mean scores) are downwardly assigned to all lower-level units (e.g., soldiers) uniquely nested in higher-level units (e.g., Army companies).
AGGREGATE.DATA(data = lq2002, grpid = "COMPID", x = c("LEAD"), aggr.stat = "mean")
names(df.combined)## [1] "COMPID" "SUB" "LEAD01" "LEAD02" "LEAD03"
## [6] "LEAD04" "LEAD05" "LEAD06" "LEAD07" "LEAD08"
## [11] "LEAD09" "LEAD10" "LEAD11" "TSIG01" "TSIG02"
## [16] "TSIG03" "HOSTIL01" "HOSTIL02" "HOSTIL03" "HOSTIL04"
## [21] "HOSTIL05" "LEAD" "TSIG" "HOSTILE" "GLEAD"
## [26] "GTSIG" "GHOSTILE" "grp.size" "aggr.model" "num.items"
## [31] "item.var" "rwg.un" "rwg.ss" "rwg.ms" "rwg.hs"
## [36] "rwg.tri" "rwg.nor" "LEAD.mn"head(df.combined)## COMPID SUB LEAD01 LEAD02 LEAD03 LEAD04 LEAD05 LEAD06 LEAD07 LEAD08 LEAD09
## 1 2 1 2 2 2 3 2 2 5 5 5
## 2 2 2 4 4 2 4 4 4 4 4 4
## 3 2 3 4 4 2 4 2 4 4 4 4
## 4 2 4 1 1 1 1 1 1 4 2 3
## 5 2 5 1 1 1 1 3 2 1 1 1
## 6 2 6 2 3 2 4 2 4 3 4 2
## LEAD10 LEAD11 TSIG01 TSIG02 TSIG03 HOSTIL01 HOSTIL02 HOSTIL03 HOSTIL04
## 1 5 4 1 5 5 0 0 0 0
## 2 4 3 3 4 3 3 2 0 0
## 3 4 2 5 5 5 1 0 0 0
## 4 3 2 1 3 2 2 1 3 3
## 5 1 1 4 4 4 2 0 0 0
## 6 4 3 4 4 4 0 0 0 0
## HOSTIL05 LEAD TSIG HOSTILE GLEAD GTSIG GHOSTILE grp.size
## 1 0 3.363636 3.666667 0.0 2.882576 3.541667 1.041667 24
## 2 1 3.727273 3.333333 1.2 2.882576 3.541667 1.041667 24
## 3 1 3.454545 5.000000 0.4 2.882576 3.541667 1.041667 24
## 4 0 1.818182 2.000000 1.8 2.882576 3.541667 1.041667 24
## 5 0 1.272727 4.000000 0.4 2.882576 3.541667 1.041667 24
## 6 0 3.000000 4.000000 0.0 2.882576 3.541667 1.041667 24
## aggr.model num.items item.var rwg.un rwg.ss rwg.ms rwg.hs rwg.tri rwg.nor
## 1 Consensus 1 1.28 0.36 0.04 -0.42 -1.91 0.03 -0.23
## 2 Consensus 1 1.28 0.36 0.04 -0.42 -1.91 0.03 -0.23
## 3 Consensus 1 1.28 0.36 0.04 -0.42 -1.91 0.03 -0.23
## 4 Consensus 1 1.28 0.36 0.04 -0.42 -1.91 0.03 -0.23
## 5 Consensus 1 1.28 0.36 0.04 -0.42 -1.91 0.03 -0.23
## 6 Consensus 1 1.28 0.36 0.04 -0.42 -1.91 0.03 -0.23
## LEAD.mn
## 1 2.882576
## 2 2.882576
## 3 2.882576
## 4 2.882576
## 5 2.882576
## 6 2.882576