Compare Samples lets you perform a test comparing the means or medians (locations) of 1 or more samples. You provide the basic components of the design (number of samples, pairing/blocking, and the normality of the distribution of the sampling population(s)). Given this information and the data, Prophet selects the test by the following scheme, based on your selection for the question "Are all samples sampled from a normal distribution?" in the Compare Samples dialog.
- If you designate that each sample is sampled from a normal distribution, then Prophet performs the appropriate normal-theory-based test (one-sample t test, paired two-sample t test, unpaired two-sample t test, one-way ANOVA, or one-way blocked ANOVA) based on the number of samples and whether or not the data are paired/blocked.
- If you designate that at least one sample is not sampled from a normal distribution, then Prophet performs the appropriate nonparametric test (one-sample signed rank test, paired signed rank test, rank sum test, Kruskal-Wallis test, or Friedman's test) based on the number of samples and whether or not the data are paired/blocked.
- If you designate that the normality of the population distributions is undetermined, then Prophet first performs a normality test for each of the samples. If one or more of the samples fails the normality test, then Prophet performs the appropriate nonparametric test (one-sample signed rank test, paired signed rank test, rank sum test, Kruskal-Wallis test, or Friedman's test) based on the number of samples and whether or not the data are paired/blocked. Otherwise, Prophet performs the appropriate normal-theory-based test (one-sample t test, paired two-sample t test, unpaired two-sample t test, one-way ANOVA, or one-way blocked ANOVA) based on the number of samples and whether or not the data are paired/blocked.
Compare Samples is a way of choosing among possible tests that may be suitable for your data. The complete list of tests that might be performed is given below. For each pair of tests listed, the first test is the one that will be performed if you specify that each sample is sampled from a normal population distribution, or if you are undecided and the normality test does not reject the null hypothesis of normality. The second test is the one that will be performed if you specify that the data are not all sampled from normal population distributions, or if you are undecided and the normality test rejects the null hypothesis of normality. For more information about a particular test, follow the appropriate link.
To properly analyze and interpret results of a Compare Samples analysis, you should be familiar with the following terms and concepts:
- one-sample problem
- paired samples
- two-sample problem
- multi-sample problem
- independent samples
- residuals
- Gaussian (normal) distribution
- equality of variance (homoscedasticity)
- violation of test assumptions
- distribution-free tests
- rank tests
- transformations
- multiplicity of testing
- multiple comparisons
- nominal vs. overall significance level
If you are not familiar with these terms and concepts, you are advised to consult with a statistician. Failure to understand and properly apply a Compare Samples analysis may result in drawing erroneous conclusions from your data. Additionally, you may want to consult the following references:
- Brownlee, K. A. 1965. Statistical Theory and Methodology in Science and Engineering. New York: John Wiley & Sons.
- Conover, W. J. 1980. Practical Nonparametric Statistics. 2nd ed. New York: John Wiley & Sons.
- Daniel, Wayne W. 1978. Applied Nonparametric Statistics. Boston: Houghton Mifflin.
- Daniel, Wayne W. 1995. Biostatistics. 6th ed. New York: John Wiley & Sons.
- Hollander, M. and Wolfe, D. A. 1973. Nonparametric Statistical Methods. New York: John Wiley & Sons.
- Lehmann, E. L. 1975. Nonparametrics: Statistical Methods Based on Ranks. San Francisco: Holden-Day.
- Miller, Rupert G. Jr. 1996. Beyond ANOVA, Basics of Applied Statistics. 2nd. ed. London: Chapman & Hall.
- Neter, J., Wasserman, W., and Kutner, M.H. 1990. Applied Linear Statistical Models. 3rd ed. Homewood, IL: Irwin.
- Sokal, Robert R. and Rohlf, F. James. 1995. Biometry. 3rd. ed. New York: W. H. Freeman and Co.
- Winer, B.J., Brown, D.R., and Michels, K.M. 1991. Statistical Principles in Experimental Design. 3rd ed. New York: McGraw Hill.
- Zar, Jerrold H. 1996. Biostatistical Analysis. 3rd ed. Englewood Cliffs, NJ: Prentice-Hall.
One-sample tests:
- t test (test whether population mean is equal to hypothesized value)
- Wilcoxon signed rank test (test whether population median is equal to hypothesized value)
Two-sample unpaired tests:
- t test (test whether two population means are equal)
- Mann-Whitney rank sum test (test whether two population distribution functions are identical against the alternative that they differ by location)
Two-sample paired tests:
- t test (test whether population mean of paired differences is 0)
- Wilcoxon signed rank test (test whether population median of paired differences is 0)
Multi-sample (one-way) unblocked tests:
- One-way ANOVA (unblocked) (test whether several treatment effects (means) are equal)
- Kruskal-Wallis test (test whether several population distribution functions are identical against the alternative that they differ by location)
Multi-sample (one-way) blocked tests:
- One-way ANOVA (blocked) (test whether several treatment effects (means) are equal.)
- Friedman's test (test whether several treatment effects (locations) are equal.)
Glossary | StatGuide Home | Home
