Intro to Marketing Research: Hypothesis Testing - Interpretation of p-Values and Effect, Sizes Cohens d Etasquared

What It Is

P-values and effect sizes are statistical measures used to evaluate the significance and practical importance of research findings in marketing research. A famous study that exemplifies the use of these measures is the "Blue Eyes/Brown Eyes" experiment by Jane Elliott, a third-grade teacher who conducted an experiment to demonstrate the effects of prejudice on behavior. In this study, Elliott divided her class into two groups based on eye color and found significant differences in behavior between the two groups. This study highlights the importance of considering both statistical significance (p-value) and practical significance (effect size) when interpreting research findings.

Key Terms & Concepts

• P-value: A statistical measure that indicates the probability of observing a result at least as extreme as the one observed, assuming that the null hypothesis is true. (e.g., a p-value of 0.05 means that there is a 5% chance of observing the result by chance).
• Null Hypothesis: A statement that there is no effect or no difference between groups. (e.g., "There is no difference in brand preference between males and females").
• Alternative Hypothesis: A statement that there is an effect or a difference between groups. (e.g., "There is a difference in brand preference between males and females").
• Effect Size: A measure of the practical importance of a research finding. (e.g., Cohen's d, Eta-squared).
• Cohen's d: A measure of effect size that indicates the standardized difference between two means. (e.g., a Cohen's d of 0.5 means that the difference between the two means is 0.5 standard deviations).
• Eta-squared: A measure of effect size that indicates the proportion of variance in the dependent variable that is explained by the independent variable. (e.g., an Eta-squared of 0.2 means that 20% of the variance in brand preference is explained by gender).
• Statistical Significance: A measure of the probability of observing a result at least as extreme as the one observed, assuming that the null hypothesis is true. (e.g., a p-value of 0.05 means that the result is statistically significant).
• Practical Significance: A measure of the practical importance of a research finding. (e.g., a Cohen's d of 0.5 means that the difference between the two means is practically significant).
• Type I Error: A false positive result, where a statistically significant result is observed when the null hypothesis is true. (e.g., a p-value of 0.05 means that there is a 5% chance of committing a Type I error).
• Type II Error: A false negative result, where a statistically insignificant result is observed when the null hypothesis is false. (e.g., a p-value of 0.05 means that there is a 5% chance of committing a Type II error).
• Power Analysis: A statistical analysis that determines the sample size required to detect a statistically significant effect. (e.g., a power analysis may indicate that a sample size of 100 is required to detect a Cohen's d of 0.5).
• Confounding Variable: A variable that is related to both the independent and dependent variables, and can affect the outcome of the study. (e.g., age may be a confounding variable in a study of brand preference).
• Regression Equation: A statistical equation that models the relationship between the independent and dependent variables. (e.g., Y = ?0 + ?1X + ?, where Y is the dependent variable, X is the independent variable, ?0 is the intercept, ?1 is the slope, and-is the error term).
• Standard Error: A measure of the variability of the sample mean. (e.g., a standard error of 0.5 means that the sample mean is 0.5 units away from the population mean).

Common Misunderstandings

• Misunderstanding: P-values are a measure of effect size.
• Correction: P-values are a measure of statistical significance, not effect size. (e.g., a p-value of 0.05 means that the result is statistically significant, but it does not indicate the size of the effect).
• Misunderstanding: A p-value of 0.05 means that the result is practically significant.
• Correction: A p-value of 0.05 means that the result is statistically significant, but it does not indicate the practical significance of the result. (e.g., a p-value of 0.05 may indicate a small effect size).
• Misunderstanding: Cohen's d is a measure of statistical significance.
• Correction: Cohen's d is a measure of effect size, not statistical significance. (e.g., a Cohen's d of 0.5 means that the difference between the two means is 0.5 standard deviations, but it does not indicate the probability of observing the result by chance).

Quick Application / Identification

Scenario: A marketing researcher conducts a study to determine whether there is a difference in brand preference between males and females. The study finds a statistically significant result, with a p-value of 0.01 and a Cohen's d of 0.8. What does this result indicate?

Answer: This result indicates that there is a statistically significant difference in brand preference between males and females, and the effect size is practically significant (Cohen's d of 0.8 means that the difference between the two means is 0.8 standard deviations).

Last?Minute Revision

• P-value is a measure of statistical significance, not effect size.
• Cohen's d is a measure of effect size, not statistical significance.
• Eta-squared is a measure of effect size that indicates the proportion of variance in the dependent variable that is explained by the independent variable.
• Type I error is a false positive result, where a statistically significant result is observed when the null hypothesis is true.
• Type II error is a false negative result, where a statistically insignificant result is observed when the null hypothesis is false.
• Power analysis is a statistical analysis that determines the sample size required to detect a statistically significant effect.
• Confounding variable is a variable that is related to both the independent and dependent variables, and can affect the outcome of the study.
• Regression equation is a statistical equation that models the relationship between the independent and dependent variables.
• Standard error is a measure of the variability of the sample mean.
• A p-value of 0.05 means that the result is statistically significant, but it does not indicate the practical significance of the result.
• A Cohen's d of 0.5 means that the difference between the two means is 0.5 standard deviations, but it does not indicate the probability of observing the result by chance.
• Eta-squared of 0.2 means that 20% of the variance in brand preference is explained by gender.
• A power analysis may indicate that a sample size of 100 is required to detect a Cohen's d of 0.5.
• A confounding variable may be age in a study of brand preference.
• A regression equation may be Y = ?0 + ?1X + ?, where Y is the dependent variable, X is the independent variable, ?0 is the intercept, ?1 is the slope, and-is the error term.
• A standard error of 0.5 means that the sample mean is 0.5 units away from the population mean. A p-value of 0.05 is not the same as a Cohen's d of 0.5. A statistically significant result does not necessarily mean that the effect is practically significant. A small effect size does not necessarily mean that the result is not practically significant. A large sample size does not necessarily mean that the result is statistically significant.