Concept Sheet 3
Probability and Uncertainty

IMPORTANT VOCABULARY

Random Experiment--an event which has at least 2 possible outcomes with uncertainty as to which will occur (Eg. picking a card from a deck of cards or selecting a sample of students from the class).

--the goal is to indicate the likelihood (probability) of each of the possible outcomes, or set of outcomes.

Sample Point/Basic Outcome--each possible result of a random experiment. (Eg. all of the possible cards in the deck or all of the possible students in the class).

Sample Space--set of all basic outcomes (denoted as S)

Examples:
S = {T,H} (for a coin flip)
S = (1,2,3,4,5,6} (for a roll of a die)
S = {all cards in a deck} (for a deck of cards)
S = {Sally, Molly, Dolly, Polly, Sue, Ann, Winnie,......., to the last student in class} (for all students in a particular class)

Event--a set of basic outcomes from the Sample Space.

Examples:
Event A might be rolling an even number on one roll of the die, or event B may be picking a King from a deck of cards, or event C may be picking a student with 2 L's in her name).

How to interpret probabilities?

1. Equally Likely Outcomes
if an experiment can yield any of N different equally likely outcomes, and exactly M of those outcomes constitute an event A, then:

p(A)=M/N.

2. Long run frequency (Poisson 1837)
The probability of an event is the proportion of times the event will occur in a large number of repetitions of the experiment.

Probability =

In other words, it is just the fraction of times a particular outcome or event (f) occurs if you were to run the experiment infinity many times.

3. Subjectivist probability
The probability of an event is the degree of confidence one has in the occurrence of the event, e.g. the probability the sun will shine tomorrow is based on individual expectations, experience, etc.

Statistical Independence--past events do no influence current or future events. For example ifyou were to flip a coin the probability of getting a "head" is 0.5. Now having gotten a "head" on the first flip, the probability of getting a "head" on the second flip is NOT different or is NOT influenced by the fact you just got a "head" on the first flip. Those who believe that events are NOT statistically independent, when indeed they are, are succumbing to the "Monte Carlo Fallacy." They believe that they are on a "roll" or have a "hot dealer" when they are just experiencing a probabalistically plausible run that could (and does) randomly occur. (Eg. getting 6 "Heads" in a row is just as likely as flipping a coin and getting, H, T, H, T, H, T.)

Probability Trees and Venn Diagrams are useful for showing probabilities visually.

Set Theory Sometimes we are interested in how more than one set of probabilities coincide. We use the language of set theory to talk about these situations.

Union--means that you are interested in the likelihood of events A OR B occurring. We denote union with the symbol .

Intersection--means that you are interested in the likelihood of events A AND B occurring at the same time. We denote intersection with the symbol .

Mutually Exclusive--two events are said to be mutually exclusive if the intersection of the two events is the null set. In other words, the two events can NOT occur at the same time; there is no intersection. If they are not mutually exclusive, they can occur at the same time and there does exist an intersection.

The following is a Venn Diagram of Events A, B, and C. A and B are mutually exclusive; they have no intersection. A and C have an intersection, as do B and C. The union of A and C is everything blue and everything red (including the sliver of C that intersects B). The union of A and B is everything Blue and everything yellow (they can have a union without intersecting).

Complement--of an event includes all of the basic outcomes NOT in a particular event, e.g. if event A is defined as spades (in a deck of cards) then the complement of A is hearts, clubs, and diamonds. In the venn diagram, the complement of A is everything not blue.

Addition Rule

We use the addition rule to find the probability of the UNION of two events.

Conditional Probability--is used when we are interested in the probability of an event GIVEN another event has already occurred. In order to calculate the conditional probability the twoevents must NOT be mutually exclusive. The best way to think about how to calculate the conditional probability is to think of the probability of A given B (keep in mind the event we are really interested in is event A). Once you know you are interested in A given B, you can forget about the rest of the sample space and only focus on the area "B." We essentially want to know how much of A is in the event B. So now we just take the fraction of A in B which is simply the ratio of the intersection of A and B to the Probability of B. Rearranging this formula we get the multiplication rule.

Multiplication Rule

We use the multiplication rule to find the INTERSECTION of two events.

From the Multiplication Rule we can see that one way to test if two events are statistically independent is if the probability of A times the probability of B is equal to the probability of the intersection. That makes good sense. Think of what it really means to be statistically independent. What it says is that events A and B are statistically independent if (for instance) the likelihood A occurring "in" B is the same likelihood of A occurring in the whole sample space.

Finally, often we are confronted with having to make decisions in an uncertain world with information that is not always perfectly reliable. Thus, we may have 2 pieces of uncertainty to deal with:
1) the inherent uncertainty about the two events we are concerned about and
2) uncertainty about how "truthful" our knowledge of the events is.

For example:
--You are interested in the stock market and want to invest. You hear about a "hot tip" from a "reliable" market analyst. You are now confronted with 2 pieces of uncertainty: the uncertainty of the stock itself and the uncertainty about the analyst's reliability.

--You are a juror on a trial and an expert gives testimony about something that happened. Then you learn that the witness is reliable only 90 percent of the time. Again two pieces to the uncertainty.

--You fear you have a terrible disease (because someone you know has it, thus there is some probability you have it). You decide to test yourself for the disease (but the test is only 97 percent reliable). Again there are two pieces to the uncertainly.

So often there are two events with uncertainly. In the last case event A would be that the patient has the disease and event B would be that the person tests "positive" for the disease. Both events occur with some probability. In order to figure out such problems we use Bayes Theorem.

Bayes Theorem

revised April 1, 1998