03 Probability

Experiments and probability models

Probability is a mathematical language for uncertainty. It assigns each event a value from $0$ to $1$: values near $0$ indicate that an event is unlikely, and values near $1$ indicate that it is likely. The same probability can be expressed as a percentage; for example, $0.25=25\%$.

A probability experiment is a process with an uncertain result. A single possible result is an outcome. The Sample space is the complete set of possible outcomes and is denoted by $S$. For a fair six-sided die,

An event is a subset of the sample space. If $A$ is the event that an even number is rolled, then

When all outcomes are equally likely, the probability of an event is

For the even-number event, $P(A)=\frac{3}{6}=\frac{1}{2}$. This favorable-outcomes ratio should be used only when the outcomes are equally likely.

Takeaway: Start every probability problem by identifying the uncertain experiment, its outcomes, and the event of interest.

Probability rules and complements

A valid probability model follows three basic rules:

1. $0\le P(A)\le 1$ for every event $A$.

2. $P(S)=1$, because some outcome in the sample space must occur.

3. If events cannot occur together, their probabilities add:

The complement of $A$, written $A^c$ or $A'$, contains every outcome that is not in $A$. The complement rule is

For example, if the probability that a package arrives late is $0.08$, then the probability that it does not arrive late is

This rule is often the quickest method for “not” questions.

Takeaway: Check that every probability is between $0$ and $1$, and use the complement rule when the desired event is the opposite of a known event.

Combining events

The intersection of $A$ and $B$, written $A\cap B$, means that both events occur. The union of $A$ and $B$, written $A\cup B$, means that at least one occurs. The general addition rule is

The intersection is subtracted because outcomes in both events would otherwise be counted twice.

For a six-sided die, let $A=\{2,4,6\}$ be the event of rolling an even number, and let $B=\{4,5,6\}$ be the event of rolling a number greater than $3$. Then $A\cap B=\{4,6\}$, so

Mutually exclusive events cannot occur together. For these events, $P(A\cap B)=0$, so the addition rule becomes

Mutual exclusivity is not the same as independence. Two nonzero-probability mutually exclusive events cannot be independent, because the occurrence of one makes the other impossible.

Takeaway: For “or,” use the addition rule and subtract the overlap unless the events are mutually exclusive.

Counting outcomes

Counting methods determine how many outcomes are possible without listing them all.

The multiplication principle states that if a process has $m$ choices for its first step and $n$ choices for its second step, then it has

possible outcomes. For a password with one letter followed by two digits, when repetition is allowed, the number of passwords is

A permutation is used when order matters. The number of ways to select and arrange $r$ objects from $n$ distinct objects is

For example, choosing a president and vice president from $10$ people gives

A combination is used when order does not matter. The number of ways to choose $r$ objects from $n$ distinct objects is

A three-person committee chosen from $10$ people can be formed in

ways.

Decision rule: Use a permutation for ordered roles or rankings. Use a combination for an unordered group or selection.

Conditional probability

Conditional probability restricts attention to outcomes in which a specified condition has already occurred. The notation $P(A\mid B)$ means the probability of $A$ given $B$. When $P(B)>0$,

Rearranging produces the multiplication rule:

It can also be written as

Suppose a fair die is rolled. Let $A$ be the event that the result is $2$ or $3$, and let $B=\{2,4,6\}$ be the event that the result is even. Once $B$ is known, only three outcomes remain possible, and just one of them belongs to $A$. Therefore,

The same result follows from the formula:

In a two-way table, the condition determines the denominator. If a group contains $52$ people, a probability calculated within that group uses $52$, not the total number of people, as its denominator.

Takeaway: For “given” questions, restrict the sample space to the condition and use the conditional event as the denominator.

Independence and dependence

Two events are independent when knowing that one occurred does not change the probability of the other. Equivalent conditions, when the probabilities are defined, include

and

For independent events, the multiplication rule becomes

For example, tossing a fair coin and rolling a fair die are independent experiments. The probability of heads and a $6$ is

Independence differs from mutual exclusivity. On one die roll, rolling a $1$ and rolling a $6$ are mutually exclusive, because both cannot occur at once. They are not independent: knowing that a $1$ occurred makes a $6$ impossible.

Sampling with replacement is often independent because the population composition stays the same. Sampling without replacement is generally dependent because each selection changes the probabilities for later selections.

Takeaway: Independence means “no change in probability,” while mutual exclusivity means “cannot occur together.”

Bayes’ theorem and evidence

Bayes’ theorem reverses the direction of a conditional probability. When $P(B)>0$, it states

If $A$ and $A^c$ partition the sample space, the denominator can be expanded using the law of total probability:

Combining these expressions gives

Consider a condition-testing example. Suppose $P(D)=0.01$, $P(+\mid D)=0.95$, and $P(+\mid D^c)=0.05$. First find the overall probability of a positive result:

Then

Thus, a positive result implies approximately a $16.1\%$ probability of the condition in this population. The prior probability and the false-positive rate matter, so $P(D\mid +)$ is not generally equal to $P(+\mid D)$.

Takeaway: Bayes’ theorem combines prior probability with new evidence and prevents the common mistake of reversing a conditional probability without adjustment.

A practical problem-solving method

A reliable solution process keeps the event definitions and denominators explicit.

1. Define the experiment and identify what is uncertain.

2. Specify the sample space or the relevant population.

3. Define the events with clear notation.

4. Decide whether the outcomes are equally likely.

5. Translate key words:

   • “not” suggests a complement;

   • “or” suggests a union and the addition rule;

   • “and” suggests an intersection and a multiplication rule;

   • “given” signals conditional probability.

6. Choose a method:

   • use permutations when order matters;

   • use combinations when order does not matter;

   • use Bayes’ theorem when reversing a conditional probability.

7. Check that the result lies between $0$ and $1$ and fits the context.

For a final check, ask whether the denominator matches the population or restricted group being described. Also check whether overlap has been subtracted in an addition problem and whether independence has actually been established before multiplying probabilities directly.

Final takeaway: Model the situation first, select the rule that matches the wording, show the relevant denominator, and test whether the result is mathematically and contextually reasonable.