3 Conditional Probability and Independence

Reading conditional probabilities

Probability answers can change when additional information becomes available. If the condition is event $B$, the relevant reference set is no longer the entire sample space but the outcomes contained in $B$. Thus, for $P(B)>0$,

The notation is read “$A$ given $B$.” The order matters: $P(A\mid B)$ generally differs from $P(B\mid A)$ because the denominators are different.

For example, suppose $40\%$ of students take statistics, $25\%$ take computer science, and $15\%$ take both. Then

whereas

Takeaway: Always identify the event after the conditioning bar and use that event’s probability as the denominator.

Building joint probabilities

The multiplication rule follows by rearranging the definition of conditional probability:

Because intersection is commutative, the same joint probability can also be written as

For a box containing 5 red balls and 3 blue balls, the probability of drawing two red balls without replacement is

The second draw has probability $\frac{4}{7}$, not $\frac{5}{8}$, because the first red ball is not replaced.

For a sequence of three events, the same idea gives

More generally, each factor conditions on all earlier events.

Takeaway: For sequential outcomes, multiply the probability of the first event by the appropriate conditional probability at each later step.

Partitions and total probability

A partition divides the sample space into cases that do not overlap and together include every possible outcome. If $B_1,\ldots,B_k$ form a partition, then any event $A$ can be split into disjoint pieces:

Adding the probabilities of these pieces and applying the multiplication rule produces the law of total probability:

Consider three factories. Factory 1 supplies $50\%$ of products with a $2\%$ defect rate, Factory 2 supplies $30\%$ with a $3\%$ defect rate, and Factory 3 supplies $20\%$ with a $5\%$ defect rate. If $D$ is the event that a product is defective, then

The overall defect probability is therefore $2.9\%$. This is a weighted average, not an unweighted average of the three defect rates.

Takeaway: When outcomes are divided into cases, calculate the contribution of each case and weight it by that case’s probability.

Reversing conditions with Bayes’ theorem

Bayes’ theorem reverses the direction of conditioning. Starting from the two forms of the joint probability,

and

equating them gives

Suppose the factory data above are known and a product is defective. The probability that it came from Factory 3 is

Factory 3 supplies only $20\%$ of products but accounts for about $34.5\%$ of defective products because its defect rate is highest. The calculation uses both the prior probability $P(F_3)$ and the conditional rate $P(D\mid F_3)$.

A useful interpretation is

Takeaway: Do not reverse a conditional probability by intuition alone; use Bayes’ theorem and include the base rate.

Testing independence

Independence means that information about one event does not change the probability of another. The central test is

When the relevant conditional probabilities exist, independence is also characterized by

or equivalently

For two tosses of a fair coin, let $A$ be “the first toss is heads” and $B$ be “the second toss is heads.” Then

Since

the events are independent. Their complements are also independent in the corresponding pairings.

By contrast, mutually exclusive events satisfy $P(A\cap B)=0$. If both have positive probability, then $P(A)P(B)>0$, so they cannot be independent. On one die roll, “the result is 1” and “the result is 2” are mutually exclusive and therefore dependent.

Takeaway: Independence is an equality to check, while mutual exclusivity describes an impossible intersection; positive-probability mutually exclusive events are not independent.

From pairwise to mutual independence

For three events, checking every pair is not always enough. Pairwise independence requires

and

Mutual independence requires these pairwise conditions and also the three-way condition

Therefore, pairwise independence is weaker than mutual independence. For a larger collection, mutual independence requires the corresponding product rule for every nonempty subcollection, not merely for pairs.

A reliable workflow for probability problems is:

1. Define each event clearly.

2. Identify the direction of conditioning.

3. Look for a partition into cases.

4. Use the multiplication rule for sequences.

5. Use Bayes’ theorem when the requested conditional probability reverses the given direction.

6. Test independence instead of assuming it.

7. Check that every probability lies between $0$ and $1$, and that probabilities over a complete partition sum to $1$.

Final takeaway: Conditional probability changes the reference set, multiplication combines successive conditions, total probability combines partitioned cases, Bayes’ theorem reverses conditioning, and independence must be established by factorization.