← Previous Contents    Next →  

Module 1 Probability

Sample spaces, events and probability

Let $S$ be the set of all possible outcomes from an experiment or similar. We call $S$ the sample space. An event is a subset $E\subseteq S$ with an assigned number $P(E)$ between $0$ and $1$, called the probability of the event.

There are essentially two intepretations of what the number $P(E)$ means.

In either of these intepretations (or any other interpretation) we must have:

From these three properties we deduce all other facts about probabilities. For instance:

As probabilties are defined using sets, everything you have learned about sets will have applications in calculating probabilities.

We will be using two examples of sample spaces to illustrate many of the concepts in the sequel, so lets familiarize ourselves with them straight away.

Dice

Consider $n$ fair dice each with $6$ sides numbered $1$ to $6$. The sample space $S$ for one roll of $n$ dice has $6^n$ elements. For $n=1$, we can list the elements of $S$ as $1,2,3,4,5,6$. For $n=2$, we can view the samples space as entries of a $6\times 6$ matrix:

For $n\geq 2$ it becomes inconvenient to list the elements of $S$, at least without a computer.

Two examples of sample spaces which are similar to the sample space for dice:

Cards

Conside a standard deck of $52$ playing cards. We can draw a hand of $k$ cards in $C(n,k)$ ways. For $k=1$, we have $52$ elements in the sample space. For $k=2$ we already have $1326$ elements in the samples space, and it would be very tedious to list them without a computer.

Two examples of sample spaces which are similar to cards:

Queues

We have $n$ people. Of these $k$ are placed in a queue. Our sample space is all possible queue placements of $k$ of the $n$ people. The size of that space is $P(n,k)$. We will mostly consider the special case where all $n$ are placed in the queue (i.e. $k=n$), when the number of queues is $P(n,n)=n!$. For $n=5$ we have $5!=120$, and it would be quite painful to list the sample space by hand.

The sample space for $n=1,2,3,4$ (people are labelled $a,b,\cdots$):

$a$
$ab,ba$
$abc, acb, bac, bca, cab, cba$
$abcd, acbd, bacd, bcad, cabd, cbad, abdc, acdb, badc, bcda, cadb, cbda, adbc, adcb, $ $ bdac, bdca, cdab, cdba, dabc, dacb, dbac, dbca, dcab, dcba$

Two examples of sample spaces which are similar to queues:

I recommend doing the quiz before continuing.

Simple sample spaces

In a simple sample space, the set $S$ is finite and all elements $x\in S$ has equal probability $$P(\{x\})=\frac{1}{|S|}.$$ The probability of an event $E$ is then $$P(E)= \frac{|E|}{|S|},$$ and calculating probabilities is reduced to a problem in enumeration.

How do we count the elements in $E$ (and $S$)?

Enumeration

I include an overview of methods for enumeration from the previous maths course (REA1101):

The four principles for counting:

When counting you should be really careful and understand why you are using the arithmetic operations $+,-,*,/$. Also, if you are unable to solve the problem, you could always try to translate the problem into a more familiar situation where you are able to count. Finally, there are always more than one way to count, and a different (but correct) approach will also give the correct answer. These principles are summarised as:

Recall also that you have learned the principle of inclusion-exclusion, $$|A\cup B| = |A|+|B|-|A\cap B|.$$

Four basic situations occur quite frequently. We phrase these in terms of picking random collections of marbles from a bag. More precisely, you have a bag of marbles numbered $1$ to $n$ and you pick out $m$ of these at random. Clearly, you will not encounter many problems which directly deals with picking marbles from a bag, but you may be able to translate your problem into that form. Once you are convinced that your problem is equivalent to picking marbles from a bag, you need to ask yourself the following two questions.

The answer to these question of course depends on the problem you are trying to solve, and you should think carefully before answering. When you know the answer, there are the four different formulas as follows.

Likely you will then need to combine these formulas in various ways using sums and products. Note that the last formula on the list will not be used in this course and can safely be forgotten for now.

A couple of remarks: First, if the enumeration problem is small, then we always list all the possibilities. Also, I should add that "small" could be fairly big, now that you are learning programming. In addition, if you are satisfied with an approximate answers, you could also consider running simulations. Simulations are a very powerful tool, which can be used for enumeration in situations where the methods you have learned here fail spectacularly. Simulations will be discussed later in this module.

Dice

We look at some examples where we rolling $n$ fair dice with $6$ sides numbers $1,2,3,4,5,6$.

What is the probability of having $2$ sixes when rolling two dice?

The sample space has $36$ elements, and only one of them is in the event space. So the probabilty is $\frac{1}{36}\approx 0.028$

What is the probability of having at least one sixe when rolling two dice?

We can view the sample space as a matrix $$\begin{pmatrix} & 1 & 2 & 3 & 4 & 5 & 6 \\ 1 & 0 & 0 & 0 & 0 & 0 & 0 \\ 2 & 0 & 0 & 0 & 0 & 0 & 0 \\ 3 & 0 & 0 & 0 & 0 & 0 & 0 \\ 4 & 0 & 0 & 0 & 0 & 0 & 0 \\ 5 & 0 & 0 & 0 & 0 & 0 & 0 \\ 6 & 0 & 0 & 0 & 0 & 0 & 0\end{pmatrix}$$ where each $0$ is an element of the sample space. Using $X$ to indicate the event space we have $$\begin{pmatrix} & 1 & 2 & 3 & 4 & 5 & 6 \\ 1 & 0 & 0 & 0 & 0 & 0 & X \\ 2 & 0 & 0 & 0 & 0 & 0 & X \\ 3 & 0 & 0 & 0 & 0 & 0 & X \\ 4 & 0 & 0 & 0 & 0 & 0 & X \\ 5 & 0 & 0 & 0 & 0 & 0 & X \\ 6 & X & X & X & X & X & X\end{pmatrix}$$ So the probability is $\frac{11}{36}\approx 0.306$.

We could also have calculated follows. 1. "At least one six" means we either have $1$ or $2$ sixes. We consider these cases separately. If we have $1$ six, then we have $2$ choices for determining which dice has a six and $5$ possible values on the other dice, giving a total of $2\cdot 5 = 10$ possibilities. There are $1$ possibility of having $2$ sixes. In total the probability is $\frac{10+1}{36}=\frac{11}{36}\approx 0.306$

What is the probability of having no sixes when we roll $2$ dice?

Using the calculation above, we get the answer is $1-\frac{11}{36}=\frac{25}{36}\approx 0.694$.

Another solution: Each dice can now have $5$ different values giving us $5\cdot 5=25$ possibilities. So the probability is $\frac{25}{36}\approx 0.694$.

What is the probability that the sum is $6$ when we roll $2$ dice?

We have the event space: $$\begin{pmatrix} & 1 & 2 & 3 & 4 & 5 & 6 \\ 1 & 0 & 0 & 0 & 0 & X & 0 \\ 2 & 0 & 0 & 0 & X & 0 & 0 \\ 3 & 0 & 0 & X & 0 & 0 & 0 \\ 4 & 0 & X & 0 & 0 & 0 & 0 \\ 5 & X & 0 & 0 & 0 & 0 & 0 \\ 6 & 0 & 0 & 0 & 0 & 0 & 0\end{pmatrix}$$ and so the probability is $\frac{5}{36}\approx 0.139$.

What is the probability of having $1$ six when we roll $4$ dice?

There are $4$ choices for determining the dice which shows $6$ and $5^3$ possible values for the remaining dice. So the probability is $\frac{4\cdot 5^3}{6^4}\approx 0.39$.

What is the probability of having $4$ six when we roll $8$ dice?

There are $C(8,4)$ possibilities for determining the dice which show $6$, and $5^4$ values for the remaining dice. So the probability is $\frac{C(8,4)\cdot 5^4}{6^8}\approx 0.026$.

What is the probability of having the dice sum to $10$ if we roll $3$ dice

As far as I am aware, there are no simple formulas for solving these types of questions with many dice. But a small number of dice is manageble.

First note that if the first two dice are roll, there is at most one number on the last dice which give us sum $10$. Let $E_i$ be the event that the first two dice sum to $i$. Then using for instance the matrix for two dice:

$P(E_1) = 0 = P(E_{13})$
$P(E_2) = \frac{1}{36} = P(E_{12})$
$P(E_3) = \frac{2}{36} = P(E_{11})$
$P(E_4) = \frac{3}{36} = P(E_{10})$
$P(E_5) = \frac{4}{36} = P(E_9)$
$P(E_6) = \frac{5}{36} = P(E_8)$
$P(E_7) = \frac{6}{36}$

The first two dice can sum to $4,5,6,7,8$ or $9$, giving us the probability $\frac{3+4+5+6+5+4}{216}=0.125$.

What is the probability of having the dice sum to $13$ if we roll $4$ dice

As above let $E_i$ be the event that the first two dice sum to $i$ and let $F_i$ be the event that the last two sum to $i$. For each $i$, there are $|F_{13-i}$ possibilities on the second dice giving sum $13$. The first two dice can show $2,3,4,5,6,7,8,9,10$ or $11$. So the probability is:

$$P(\text{sum is $13$}) = \frac{\sum_i |E_i|\cdot |F_{13-i}|}{6^4}$$ $$ = \frac{1\cdot 2 + 2\cdot 3 + 3\cdot 4 + 4 \cdot 5 + 5\cdot 6 + 6\cdot 5 + 5\cdot 4 + 4 \cdot 3 + 3 \cdot 2 + 2\cdot 1 }{6^4}$$ $$= \frac{140}{6^4}\approx 0.108.$$

Cards

We discuss some examples with $m=5$ cards drawn from a standard deck of $52$ cards. For cards we may calculate ordered or unordered, as long as both the event space and the sample space is calculated in the same way. With ordered selection the sample space has $$|S|=P(52,5)$$ elements, and otherwise we get $$|S|=C(52,5).$$ In the examples below we will work with the unordered sample space.

What is the probability that we have $4$ aces?

One of the cards is not an ace. There are $52-4=48$ choices for determining this card. The other cards must all be aces, so the probability is $$\frac{48}{C(52,5)}\approx 1.85\cdot 10^{-5}.$$

What is the probability that we have $2$ aces and $3$ jacks.

We select $2$ out of $4$ aces and $3$ out of four jacks and get the probability $$\frac{C(4,2)\cdot C(4,3)}{C(52,5)}\approx 9.23\cdot 10^{-6}.$$

What is the probability that all the cards have the same suit?

We make two selections. First we select the suit: $C(4,1)$ and then the four cards in that suit C(13,5). Finally, we multiply and get the answer $$\frac{C(4,1)C(13,5)}{C(52,5)}\approx 1.98\cdot 10^{-3}$$

What is the probability of getting one pair?

There are $13\cdot C(4,2)$ possibilities for choosing the pair. The remaining cards can be chosen in $C(12,3)\cdot 4^3$ ways. This gives the probability $$P=\frac{13\cdot C(4,2)\cdot C(12,3)\cdot 4^3}{C(52,5)}\approx 0.42.$$

Queues

We discuss some examples where we place $6$ people in a queue. We label the people $a,b,c,d,e,f$.

What is the probability that the queuing order will be $fedcba$?

That would be $\frac{1}{6!}$ since the sample space has $6!$ elements.

What is the probability that $a$ is at the front?

We can place $b,c,d,e,f$ freely at positions $2,3,4,5$, giving us the answer $\frac{5!}{6!}=\frac{1}{6}.$

What is the probability that $a$ is not at the back?

We can place $a$ in $5$ places and $b,c,d,e,f$ freely. The answer is $\frac{5\cdot 5!}{6!}=\frac{5}{6}$.

We could also have used the previous answer and computed $1-\frac{1}{6}=\frac{5}{6}$.

What is the probability that $a$ is in front of $b$?

If $a$ is at the front, then $b$ can be placed in $5$ positions. If $a$ is at position $2$, then $b$ can be placed in $4$ positions, and so on. The remaining people can be places in $4!$ positions. The answer becomes $\frac{(5+4+3+2+1)\cdot 3!}{6!}=\frac{1}{2}.$

An alternative solution is as follows: We choose two positions for $a$ and $b$, and place $a$ at the front. This can be done in $C(6,2)$ ways. We get $\frac{C(6,2)\cdot 4!}{6!}=\frac{1}{2}$ as above.

What is the probability that $a$ and $b$ occupy the first two positions (in any order)?

There are $2$ possibilites for placing $a$ and $b$. The others can be placed in $3!$ ways, giving us the answer $\frac{1}{60}$.

What is the probability that $a$ and $b$ are both in front of $c$ and $d$?

The last position occupied by $a$ and $b$ can be $2,3$ or $4$, with respectively $2,6$ and $12$ ways to place $a$ and $b$, and $12, 6$ and $2$ ways to place $c$ and $d$. For all of these, there are $2$ ways to place $e$ and $f$. The answer becomes $\frac{(2\cdot 12+6\cdot 6+12\cdot 2)\cdot 2}{6!}=\frac{7}{30}$.



← Previous Contents    Next →