The Chi-Square Distribution

Introduction

Key Concepts

Definition of Chi-Square Distribution

The Chi-Square Distribution is a continuous probability distribution that arises from the sum of the squares of independent standard normal random variables. It is denoted as χ² and is characterized by its degrees of freedom (df), which determine its shape. The Chi-Square Distribution is always non-negative and is skewed to the right, with the degree of skewness decreasing as the degrees of freedom increase.

Degrees of Freedom

Degrees of Freedom (df) in the context of the Chi-Square Distribution refer to the number of independent values that can vary in the analysis without violating any given constraints. It plays a crucial role in determining the specific Chi-Square Distribution used in hypothesis testing.

For example, in a Chi-Square test for independence in a contingency table, the degrees of freedom are calculated as: $$ df = (r - 1) \times (c - 1) $$ where \( r \) is the number of rows and \( c \) is the number of columns in the table.

Goodness of Fit Test

The Goodness of Fit test evaluates whether a set of observed frequencies matches a set of expected frequencies based on a particular hypothesis. It determines how well the observed data fit the expected distribution.

The test statistic is calculated as: $$ \chi^2 = \sum \frac{(O_i - E_i)^2}{E_i} $$ where \( O_i \) is the observed frequency and \( E_i \) is the expected frequency for the \( i^{th} \) category.

A higher Chi-Square statistic indicates a greater discrepancy between observed and expected frequencies, suggesting that the null hypothesis may be rejected.

Test for Independence

The Test for Independence assesses whether two categorical variables are independent of each other in a population. It is commonly used in contingency tables to examine the relationship between variables.

The Chi-Square statistic for this test is calculated similarly: $$ \chi^2 = \sum \frac{(O_{ij} - E_{ij})^2}{E_{ij}} $$ where \( O_{ij} \) is the observed frequency and \( E_{ij} \) is the expected frequency for the cell in the \( i^{th} \) row and \( j^{th} \) column.

Degrees of freedom for the Test for Independence are calculated as: $$ df = (r - 1) \times (c - 1) $$

Assumptions of Chi-Square Tests

For Chi-Square tests to be valid, certain assumptions must be met:

• Data must be in the form of frequencies or counts of cases.
• Categories are mutually exclusive and exhaustive.
• Expected frequency for each category should be at least 5 to ensure the approximation to the Chi-Square Distribution is accurate.
• Observations are independent of each other.

Calculating Expected Frequencies

Expected frequencies are crucial for both Goodness of Fit and Test for Independence. They represent the frequencies expected under the null hypothesis.

For the Goodness of Fit test: $$ E_i = N \times p_i $$ where \( N \) is the total number of observations and \( p_i \) is the expected proportion for category \( i \).

For the Test for Independence in a contingency table: $$ E_{ij} = \frac{(Row_i \ Total) \times (Column_j \ Total)}{Grand \ Total} $$

P-value and Hypothesis Testing

After calculating the Chi-Square statistic, the next step is to determine the p-value, which helps in deciding whether to reject the null hypothesis. The p-value is the probability of observing a Chi-Square statistic as extreme as, or more extreme than, the calculated value under the null hypothesis.

To interpret the p-value:

• If \( p \leq \alpha \) (significance level), reject the null hypothesis.
• If \( p > \alpha \), fail to reject the null hypothesis.

Applications of Chi-Square Distribution

The Chi-Square Distribution has diverse applications in various fields:

• Genetics: Testing ratios of genetic traits.
• Marketing: Analyzing consumer preferences across different categories.
• Quality Control: Assessing the distribution of defects in manufacturing.
• Social Sciences: Evaluating relationships between categorical variables.

Advantages of Using Chi-Square Tests

• Non-parametric: Does not assume a normal distribution of the data.
• Versatile: Applicable to various types of categorical data.
• Simple to compute and interpret, especially with contingency tables.

Limitations of Chi-Square Tests

• Requires a sufficiently large sample size to ensure validity.
• Cannot be used with small expected frequencies (ideally all \( E_i \geq 5 \)).
• Sensitive to sample size; large samples may detect trivial differences as significant.

Example Problem: Goodness of Fit

Suppose a die is rolled 60 times, and the observed frequencies for each face are as follows:

• 1: 8
• 2: 12
• 3: 10
• 4: 10
• 5: 10
• 6: 10

We want to test if the die is fair at a significance level of 0.05.

Step 1: Define Hypotheses

• Null Hypothesis (\( H_0 \)): The die is fair; each face has an equal probability of \( \frac{1}{6} \).
• Alternative Hypothesis (\( H_A \)): The die is not fair.

Step 2: Calculate Expected Frequencies

$$ E_i = \frac{60}{6} = 10 \quad \text{for each face} $$

Step 3: Compute Chi-Square Statistic

$$ \chi^2 = \sum \frac{(O_i - E_i)^2}{E_i} = \frac{(8-10)^2}{10} + \frac{(12-10)^2}{10} + 4 \times \frac{(10-10)^2}{10} = \frac{4}{10} + \frac{4}{10} + 0 = 0.8 $$

Step 4: Determine Degrees of Freedom

$$ df = k - 1 = 6 - 1 = 5 $$

Step 5: Find P-value

Using the Chi-Square distribution table, \( \chi^2 = 0.8 \) with \( df = 5 \) yields a p-value greater than 0.05.

Step 6: Decision

Since \( p > 0.05 \), we fail to reject the null hypothesis. There is insufficient evidence to conclude that the die is unfair.

Example Problem: Test for Independence

Consider a sample of 200 students surveyed to determine if there is an association between gender and preference for online versus in-person classes. The contingency table is as follows:

	Online	In-Person	Total
Male	60	40	100
Female	80	20	100
Total	140	60	200

We aim to test the independence of gender and class preference at a 0.05 significance level.

Step 1: Define Hypotheses

• Null Hypothesis (\( H_0 \)): Gender and class preference are independent.
• Alternative Hypothesis (\( H_A \)): Gender and class preference are not independent.

Step 2: Calculate Expected Frequencies

For Male-Online: $$ E = \frac{(100 \times 140)}{200} = 70 $$ For Male-In-Person: $$ E = \frac{(100 \times 60)}{200} = 30 $$ For Female-Online: $$ E = \frac{(100 \times 140)}{200} = 70 $$ For Female-In-Person: $$ E = \frac{(100 \times 60)}{200} = 30 $$

Step 3: Compute Chi-Square Statistic

$$ \chi^2 = \sum \frac{(O - E)^2}{E} = \frac{(60-70)^2}{70} + \frac{(40-30)^2}{30} + \frac{(80-70)^2}{70} + \frac{(20-30)^2}{30} = \frac{100}{70} + \frac{100}{30} + \frac{100}{70} + \frac{100}{30} \approx 1.4286 + 3.3333 + 1.4286 + 3.3333 = 9.523 $$

Step 4: Determine Degrees of Freedom

$$ df = (r - 1) \times (c - 1) = (2 - 1) \times (2 - 1) = 1 $$

Step 5: Find P-value

Using the Chi-Square distribution table, \( \chi^2 = 9.523 \) with \( df = 1 \) yields a p-value less than 0.05.

Step 6: Decision

Since \( p < 0.05 \), we reject the null hypothesis. There is significant evidence to suggest that gender and class preference are not independent.

Comparison Table

Aspect	Goodness of Fit	Test for Independence
Purpose	Assess if observed frequencies match expected frequencies based on a specific distribution.	Determine if there is an association between two categorical variables.
Application	Single categorical variable.	Two categorical variables in a contingency table.
Degrees of Freedom	Number of categories minus one (\( k - 1 \)).	(\( r - 1 \)) \times (\( c - 1 \)).
Pros	Simple to perform; useful for distribution fitting.	Effective in identifying relationships between variables.
Cons	Requires large expected frequencies; only applicable to categorical data.	Similar limitations as Goodness of Fit; cannot specify the nature of the association.

Summary and Key Takeaways