Lecture 4d

Conditional probabilities

Andrew Pua

2024-10-02

Plan for these slides

  1. Moving on to notion of dependence and independence
  2. Updating probabilities

How the receipt of new information affects probability calculations

Consider the situation where two fair dice are tossed and the outcomes on both faces are recorded. Each outcome is assumed to be equally likely to occur.

  1. What is the sample space?
  2. Define A as the event where a sum of 10 shows up and B as the event where the faces of the two dice are not identical. What are \(\mathbb{P}\left(A\right)\) and \(\mathbb{P}\left(B\right)\)?
  1. Suppose we know or could imagine that \(B\) occurred. What will be the new sample space?
  2. How will the probability of \(A\) be affected by \(B\) occurring?
  3. Suppose we know or could imagine that \(A\) occurred. What will be the new sample space?
  4. How will the probability of \(B\) be affected by \(A\) occurring?

Definition of conditional probability

Definition 1 (Wasserman (2004, p. 10) Definition 1.12) If \(\mathbb{P}\left(B\right)>0\), then the conditional probability of \(A\) given \(B\) is \[\mathbb{P}\left(A|B\right)=\frac{\mathbb{P}\left(AB\right)}{\mathbb{P}\left(B\right)}.\]

Being careful with conditional probabilities

  1. Conditional probabilities are indeed probabilities. There are constraints on values they could take.
  2. \(A|B\) is NOT a set operation.
  3. Mathematically, \(\mathbb{P}\left(A|B\right) \neq \mathbb{P}\left(B|A\right)\). Yet, many confuse these two, which leads to the so-called prosecutor’s fallacy.
  1. Let \(B\) be a fixed event with \(\mathbb{P}\left(B\right)>0\). The function \(A\mapsto \mathbb{P}\left(A|B\right)\) is a probability function.
  2. So, for disjoint events \(A\) and \(C\), \[\mathbb{P}\left(A\cup C|B\right)=\mathbb{P}\left(A|B\right)+\mathbb{P}\left(C|B\right).\]
  3. DO NOT EVER DO THIS: \[\mathbb{P}\left(A|B\cup C\right)=\mathbb{P}\left(A|B\right)+\mathbb{P}\left(A|C\right).\]

Definition is quite powerful!

We can conclude from the definition that \[\mathbb{P}\left(B|A\right)=\frac{\mathbb{P}\left(A|B\right)\mathbb{P}\left(B\right)}{\mathbb{P}\left(A\right)}\]

  • This is one version of Bayes’ Theorem. Refer to Theorem 1.17 in Wasserman (2004, p. 12).
  • The formula provides a “disciplined” approach of transforming \(\mathbb{P}\left(B\right)\) to \(\mathbb{P}\left(B|A\right)\) after getting information that \(A\) occurred.

  • In the previous context, \(\mathbb{P}\left(B\right)\) is sometimes referred to as the prior probability and \(\mathbb{P}\left(B|A\right)\) is sometimes referred to as the posterior probability.

  • In addition, we can also use the definition to obtain the Law of Total Probability. Refer to Wasserman (2004, p. 12) Theorem 1.16.

  • The idea starts from \(A=(AB) \cup (AB^c)\). Then apply the definition of conditional probability to obtain \[\mathbb{P}\left(A\right)=\mathbb{P}\left(A|B\right)\mathbb{P}\left(B\right)+\mathbb{P}\left(A|B^c\right)\mathbb{P}\left(B^c\right)\]

Question from Statistical Literacy of MPs

Suppose there was a diagnostic test for a virus. The false-positive rate (the proportion of people without the virus who get a positive result is one in 1000. You have taken the test and tested positive. What is the probability that you have the virus?

COVID19 testing

Consider this article about COVID19 around 2020.

Work out the calculations for:

  1. The true positive rate (sometimes called specificity)
  2. The true negative rate (sometimes called sensitivity)
  3. The probability of an incorrect test result
  4. The probability of a positively tested person being actually infected
  5. The probability of a negatively tested person being not infected

Defining independence

Definition 2 (Wasserman (2004, p. 8) Definition 1.9) Two events \(A\) and \(B\) are independent if \[\mathbb{P}\left(AB\right)=\mathbb{P}\left(A\right)\mathbb{P}\left(B\right).\]

What is the connection to the definition of conditional probability? Refer to Lemma 1.14.

Do not confuse independence with disjointness.

  1. \(A\) and \(B\) are disjoint is the same as saying \(\mathbb{P}\left(AB\right)= 0\).
  2. \(A\) and \(B\) are independent is the same as saying \(\mathbb{P}\left(AB\right)=\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)\).

Why do students confuse the two?

  1. Toss a fair die. Let \(A=\{2,4,6\}\) and \(B=\{1,2,3,4\}\). What are \(\mathbb{P}\left(A\right)\), \(\mathbb{P}\left(B\right)\), \(\mathbb{P}\left(AB\right)\)?

  2. Toss a fair die. Let \(A=\{2,4,6\}\) and \(B=\{1,3,5\}\). What are \(\mathbb{P}\left(A\right)\), \(\mathbb{P}\left(B\right)\), \(\mathbb{P}\left(AB\right)\)?

Independence is convenient, but …

Suppose Ronald and Robert are students in a statistics course. Define \(A\) as the event that Ronald passes the course and \(B\) as the event that Robert passes the course.

Suppose \(\mathbb{P}\left(A\right)=0.9\) and \(\mathbb{P}\left(B\right)=0.7\). What is the probability that both pass the course if

  1. independence of \(A\) and \(B\) is assumed?
  2. if \(\mathbb{P}\left(B|A\right)=0.5\)?
  3. if \(\mathbb{P}\left(B|A\right)=0.9\)?

Independence is convenient, but …

Problem 1.32 of Arias-Castro: Suppose we flip a coin three times in sequence. Assume that any outcome is equally likely. What is the probability that the last toss lands heads if the previous tosses landed heads?

Problem 1.32 of Arias-Castro: Suppose we flip a coin three \(n\geq 2\) times in sequence. Assume that any outcome is equally likely. What is the probability that the last toss lands heads if the previous tosses landed heads?

What happens when there are more than 2 events?

Definition 3 (Wasserman (2004, p. 8) Definition 1.9) Let \(I\) be an index set. A set of events \(\{A_i: i\in I\}\) is independent if

\[\mathbb{P}\left(\bigcap_{i\in J}A_i\right)=\prod_{i\in J}\mathbb{P}\left(A_i\right)\] for every finite subset \(J\) of \(I\).

Subtle aspect of definition of independence

  • Consider the case of tossing 2 fair dice. Assume that the outcomes of the tosses are equally likely.

  • Let \(A\) be the event that the outcome of the first die is even.

  • Let \(B\) be the event that the outcome of the second die is even.

  • Let \(C\) be the event that the sum of the outcomes of the both dice is even.

Two things to do:

  1. Show that \(A\) and \(B\) are independent, \(A\) and \(C\) are independent, \(B\) and \(C\) are independent, but \[\mathbb{P}\left(ABC\right)\neq \mathbb{P}\left(A\right)\mathbb{P}\left(B\right)\mathbb{P}\left(C\right).\]
  2. Spinoff of Wasserman Chapter 1 Exercise 23: Design a Monte Carlo simulation for this situation.

Wasserman Chapter 1 Exercise 19

  • Suppose that 30 percent of owners use MacOS, 50 percent use Windows, and 20 percent use Linux.

  • Suppose that 65 percent of MacOS users have succumbed to a computer virus, 82 percent of Windows users get the virus, and 50 percent of the Linux users get the virus.

  • We select a person at random and learn that her system was infected by a virus. What is the probability that she is a Windows user?