Basic Probability Concepts and Rules

Introduction

Key Concepts

1. Probability Definition

Probability quantifies the likelihood of an event occurring. It is expressed as a number between 0 and 1, where 0 indicates impossibility and 1 signifies certainty. The probability of an event \( A \) is denoted as \( P(A) \).

Mathematically, if there are \( n \) equally likely outcomes and \( m \) favorable outcomes for event \( A \), then the probability of \( A \) is:

$$P(A) = \frac{m}{n}$$

2. Sample Space and Events

The sample space, denoted as \( S \), is the set of all possible outcomes of a random experiment. An event is a subset of the sample space.

• Example: Rolling a six-sided die. The sample space is \( S = \{1, 2, 3, 4, 5, 6\} \).
• Event: Rolling an even number \( E = \{2, 4, 6\} \).

3. Complementary Events

The complement of an event \( A \), denoted as \( A' \), consists of all outcomes in the sample space that are not in \( A \). The sum of the probabilities of an event and its complement is always 1:

$$P(A) + P(A') = 1$$

4. Mutually Exclusive Events

Two events are mutually exclusive (or disjoint) if they cannot occur simultaneously. If events \( A \) and \( B \) are mutually exclusive, then:

$$P(A \text{ and } B) = 0$$

5. Independent Events

Events \( A \) and \( B \) are independent if the occurrence of one does not affect the occurrence of the other. For independent events:

$$P(A \text{ and } B) = P(A) \times P(B)$$

6. Conditional Probability

Conditional probability refers to the probability of event \( A \) occurring given that event \( B \) has already occurred. It is denoted as \( P(A|B) \) and is calculated as:

$$P(A|B) = \frac{P(A \text{ and } B)}{P(B)}$$

7. Addition Rule

The addition rule is used to find the probability of the union of two events. For any two events \( A \) and \( B \), the probability of \( A \) or \( B \) occurring is:

$$P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$$

If \( A \) and \( B \) are mutually exclusive, then \( P(A \text{ and } B) = 0 \), and the formula simplifies to:

$$P(A \text{ or } B) = P(A) + P(B)$$

8. Multiplication Rule

The multiplication rule is used to determine the probability of the intersection of two events. For dependent events, it is given by:

$$P(A \text{ and } B) = P(A) \times P(B|A)$$

For independent events, since \( P(B|A) = P(B) \), it simplifies to:

$$P(A \text{ and } B) = P(A) \times P(B)$$

9. Permutations and Combinations

Permutations and combinations are methods for counting the number of possible arrangements or selections from a set.

• Permutations: Used when order matters. The number of permutations of \( n \) items taken \( r \) at a time is:
$$P(n, r) = \frac{n!}{(n - r)!}$$
• Combinations: Used when order does not matter. The number of combinations of \( n \) items taken \( r \) at a time is:
$$C(n, r) = \frac{n!}{r!(n - r)!}$$

Example: The number of ways to choose 3 committee members from 5 candidates.

Using combinations:

$$C(5, 3) = \frac{5!}{3!(5 - 3)!} = \frac{120}{6 \times 2} = 10$$

10. Probability Distributions

A probability distribution assigns probabilities to different outcomes of a random variable. It can be discrete or continuous.

• Discrete Probability Distribution: Deals with discrete random variables, which have countable outcomes.
• Continuous Probability Distribution: Deals with continuous random variables, which have uncountable outcomes.

11. Expected Value

The expected value (or expectation) of a random variable is the long-term average value of repetitions of the experiment. For a discrete random variable \( X \), it is calculated as:

$$E(X) = \sum [x \times P(x)]$$

12. Variance and Standard Deviation

Variance measures the spread of a set of values. For a discrete random variable \( X \), variance \( \sigma^2 \) is:

$$\sigma^2 = \sum [ (x - E(X))^2 \times P(x) ]$$

Standard deviation \( \sigma \) is the square root of the variance:

$$\sigma = \sqrt{\sigma^2}$$

13. Binomial Probability

The binomial probability formula calculates the probability of having exactly \( k \) successes in \( n \) independent Bernoulli trials (each with success probability \( p \)). It is given by:

$$P(k) = C(n, k) \times p^k \times (1 - p)^{n - k}$$

14. Normal Distribution

The normal distribution is a continuous probability distribution characterized by its mean \( \mu \) and standard deviation \( \sigma \). It is symmetric about \( \mu \) and described by the probability density function:

$$f(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{ -\frac{(x - \mu)^2}{2\sigma^2} }$$

15. Central Limit Theorem

The Central Limit Theorem states that the distribution of the sample mean approaches a normal distribution as the sample size becomes large, regardless of the original distribution of the population. This theorem is fundamental in inferential statistics.

Comparison Table

Concept	Definition	Application
Permutations	Arrangement of objects where order matters.	Calculating the number of possible orderings in a race.
Combinations	Selection of objects where order does not matter.	Choosing committee members from a group of candidates.
Independent Events	Events whose occurrence does not affect each other.	Flipping a coin and rolling a die.
Mutually Exclusive Events	Events that cannot occur at the same time.	Drawing a card that is both red and black.
Binomial Probability	Probability of a given number of successes in a sequence of trials.	Calculating the likelihood of getting a certain number of heads in coin tosses.
Normal Distribution	A symmetric probability distribution characterized by mean and standard deviation.	Modeling heights of individuals in a population.

Summary and Key Takeaways