Excellent article to read:
Of particular note are the sections “Independence of Events” and “Law of Averages, and the Law of Large Numbers”.
A basic assumption in probability theory is that each event is independent of all other events. That is, previous draws have no influence on the next draw. A popular catch phrase is “the dice have no memory.” A die or roulette ball cannot look back and determine that it is due for a 6 or some other number. How could a coin decide to turn up a head after 20 tails? Each event is independent and therefore the player can never predict what will come up next. If a fair coin was flipped 5 times and came up heads 5 times in a row, the next flip could be either heads or tails. The fact that heads have come up 5 times in a row has no influence on the next flip. It is wise not to treat something that is very very unlikely as if it were impossible (see Turner, 1998). In fact, if a coin is truly random, it must be possible for heads to come up 1 million times in a row. Such an event is extraordinarily unlikely, p = 1/21,000,000, but possible. Even then, the next flip is just as likely to be heads as it is tails. Nonetheless, many people believe that a coin corrects itself; if heads comes up too often, they think tails is due.
Part of the explanation for the persistent belief among those who gamble that there are patterns in chance, may stem from a misunderstanding of two related “laws” of statistics: the law of averages and the law of large numbers. The first is an informal folk theory of statistics; the second is a statistical law. These laws can be summarized as follows:
Law of Averages: Things average out over time.
Law of Large Numbers: As the sample size increases the average of the actual outcomes will more closely approximate the mathematical probability.
The law of large numbers is a useful way to understand betting outcomes. A coin on average will come up heads 50% of the time. It could nonetheless come up heads 100% of the time or 0% of the time. In a short trial, heads may easily come up on every flip. The larger the number of flips, however, the closer the percentage will be to 50%.