Tukey test, also known as Tukey’s Honest Significant Test (HSD) test, is a post-hoc statistical test used to determine whether the means of two sets of data are statistically different from each other. This test is based on the studentized range distribution and is performed after an ANOVA test has indicated a significant difference in means of three or more sets of data.

Tukey test is used when the sample sizes of all the groups are equal. If the groups have different sample sizes, an adjustment of the Tukey test, called the Tukey-Kramer test, is used instead. In this article, we will see how to perform the Tukey-Kramer test in Excel and Google Sheets. To learn about how to use the Tukey test, see the article on the Tukey test.

When to Perform Tukey-Kramer Test

Tukey-Kramer Test is performed to find out the specific pair(s) of groups that cause the difference after an ANOVA Test had shown that there is a significant difference in the means of the groups tested.

Note that the Tukey-Kramer test gives reduces to the Tukey test when the sample sizes of the groups are equal. That is, the Tukey-Kramer test and the Tukey test will return the same values when the sample sizes of the groups are equal.

How to Perform Tukey-Kramer Test in Excel

Using the q Critical Values Table

We will use the same example as used in the Tukey Test article here, but we will modify the data so that the groups do not have the same sample size.

Example

Background: A researcher wants to find out the effectiveness of three weight-loss therapies: pharmaceutical medicine approach, natural herbs approach, and combination of pharmaceutical medicine and natural herbs. He randomly selects participants for each group and administers one therapy to each of the groups. He measured the weights of the participants before the therapy and after 60 days since the commencement of the therapy. The amount of weight lost, in pounds, for each of the participants in the groups is presented in the table below. Test at a 5% significant level whether there is a significant difference in the amount of weight lost for each pair of therapies.

For the purpose of this test, we assume that the populations from which these samples were drawn have equal variances (or you can test for equal variances using Bartlett’s Test)

First, perform an ANOVA Test on the data as follows:

Use the “Anova: Single Factor” feature located in the Data Ribbon > Data Analysis.

If you don’t see the Data Analysis option, you will need to install the Data Analysis Toolpak.

In the Data Analysis window, choose Anova: Single Factor and click OK.

Configure the parameters as follows.

• Click on “Input Range” and select the cells containing the source data.
• In the “Grouped By” section, select the “Columns” radio box.
• Select the “Labels in first row” checkbox if the selected range contains column headers as shown in the example below.
• Select the desired output option. To get the results on the same sheet, select the Output range and specify the specific reference to the cell into which to display the matrix. In our case it is $E$3

The output for the ANOVA test is as follows:

From the ANOVA Test output above, you can see that the p-value is 0.0012 which is less than our significance level of 5% or 0.05. This implies that at least one of the means of the groups is significantly different from the others. However, we do not know the group(s) that contributes to this difference, hence the need for Tukey’s HSD Test. And since the sample sizes of the groups are not equal, we use Tukey-Kramer’s procedure.

<<Learn more on How to Perform an ANOVA Test here.>>

The Tukey-Kramer’s Test is performed as follows:

First, set up the groups in pairs.

Next, obtain the absolute values (positive values) of the difference in the means of each pair using the ABS function.

You can get the averages (means) for each group in the “SUMMARY” section of the ANOVA Test result.

Complete the absolute differences for the rest of the group pairs.

Next, calculate the Standard Error (SE) from the ANOVA output.

The Standard Error (SE) is giving by the formula

Where n₁ and n₂ are the sample sizes of the groups in each pair. MS_E can be found under the MS column, in the Within Groups row of the ANOVA section of the ANOVA test result.

Complete the standard error for the rest of the group pairs.

Next, we calculate the q score for each of the pairs. The q score is given by the formula:

Complete the same for the rest of the pairs.

Finally, compare the calculated q_tukey scores above with the q critical value for the given number of groups, degree of freedom, and significance level.

Here, the number of groups is 3 and the degree of freedom of the denominator is 25 as can be seen under the column in the Within Groups row of the ANOVA section of the ANOVA test result. An excerpt of the q table for 5% significance level is shown below:

From the table, you can see that the critical level for 5% significant level, 3 groups, and degree of freedom of the denominator of 25 is 3.523.

Comparing this value with the q_tukey scores obtained, you can see that there is a significant difference between the mean weight loss of those placed in pharmaceutical medicines therapy and combined pharmaceutical and natural herbs therapies because its q score of 5.94 is greater than the q critical level.

However, there is no significant difference between the mean weight loss of those in the pharmaceutical medicines therapy and natural herbs therapy and between the mean weight loss of those in the natural herbs therapy and the combined pharmaceutical and natural herbs therapies because their q scores of 2.98 and 3.34 respectively are less than the q critical level of 3.52

Using 3rd Party Excel Add-ins

Excel does not have a specialized function for performing Tukey-Kramer’s test. However, there are some available third-party Add-ins with the capability of performing Tukey-Kramer’s test. Some are free while others are premium. One of such Add-ins that is good and free to use is the Xrealstats Add-in from Real Statistics. Xrealstats add-in not only adds additional functions to the in-built Excel functions but also has a Data Analysis Tool.

For instructions on how to download and install Xrealstats add-in, visit the Real Statistics website.

We perform the Tukey-Kramer’s test on our Weight Loss over 60 Days example using the Xrealstats add-in as follows:

First, select the Real Statistics “Data Analysis Tools” located in Add-Ins Ribbon > Real Statistics.

In the Real Statistics window, select the Anova tab and then One Factor Anova and then click OK.

Next, you will need to configure the parameters.

• Click on “Input Range” and select the cells containing the source data.
• Select the “Column headings included with data” checkbox if the selected range contains column headers as shown in the example below.
• Enter the desired significance level in the “Alpha” box. In our case, alpha is 0.05.
• Select the “ANOVA” checkbox in the Omnibus test options section.
• Check the “Tukey HSD” checkbox in the ANOVA follow-up options section.
• Enter the range or the starting cell for the output in the “Output Range” box.
• Then, click “OK”.

The output is as follows:

From the output, you can see that the critical level (q-crit) is 3.523, which when compared to the q-stat for the different pairs, you can see that the q-stat for the Pharmaceutical and Combined pair is 5.94106, which is greater than q-crit, hence is significant.

Conclusion on the pairs that have significantly different means can also be made using the p-value (shown in pink). You can see that the p-value for the Pharmaceutical and Combined pair is 0.00083 which is less than alpha of 0.05, hence is significant.

 

Tukey-Kramer Test in Google Sheets

Tukey-Kramer test can be calculated in Google Sheets as follows.

Firstly, using the XLMiner Analysis Toolpak Add-on, perform an ANOVA Test on the data as follows:

After you have installed the XLMiner Analysis Toolpak add-on, you can find it in the Add-ons tab. Select XLMiner Analysis ToolPak and click on Start.

In the XLMiner Analysis ToolPak window, choose Anova: Single Factor.

In the Anova: Single Factor option dropdown, configure the parameters as follows.

• Select the cells containing the source data and click the “Input Range” box.
• In the “Grouped By” section, select the “Columns” radio box.
• Select the “Labels in first row” checkbox if the selected range contains column headers as shown in the example below.
• Enter the desired output start cell or range in the “Output Range” box.
• Then click “OK”.

The output is as follows:

The next steps are the same as illustrated previously using Excel. The final output is shown below.