gambler's fallacy

[kaysixteen]

So I am reading this *generally* excellent book on the lack of critical thinking and logical reasoning skills, written by the Irish physicist and science journalist David Robert Grimes, and he devotes an entire section of chapters to various thinking errors related to our 'collective innumeracy' as a society.  He is certainly not wrong to say this-- we are probably more innumerate as a society than illiterate, which is saying something.  And I do not claim to be someone who is particularly good at math, it always having been my worst academic subject.   I have also never had any statistics training.   Many of his claims in this section seem pretty solid, but I am often having to plead ignorant of the math bases needed to really evaluate them-- that would perhaps be another question, namely should hss reorder their college prep math sequences to place a general stats class in hs?.   But one 'fallacy' he explains has got me flummoxed, at least wrt the example he cited (and he noted that when the example was given, with the so-called fallacy explained by the professional smart person Marilyn vos Savant, numerous PhD types, including STEM ones, criticized her answer, and were, by his reckoning, agreeing with vos Savant, wrong to do so).   This is the so-called 'gambler's fallacy', the example of which was a hypothetical experience on 'Let's Make a Deal', where the contestant knows that of the three doors to choose from, A has a car, and B and C have goats.   He chooses A.   Monte Hall opens C, reveals the goat there.   Then he gives contestant the chance to change his choice from A to B.  So the question woudl be, what is the chance he would be right to make that choice?   It seems like it should be 50%, since he does not know which of A or B has the car?   But the author, following vos Savant, engages in some rather complex statistical analysis that, well... suggests that this answer is wrong.   I am not competent to judge these stat analyses, but remain unconvinced.   But I am willing to be convinced....?

[Liquidambar]

To help convince you...  Imagine there are 100 doors instead of just 3.  You pick one of them.  Monty Hall then opens 98 doors, revealing 98 goats.  There are only 2 doors still closed:  the one you picked, and the one Monty Hall left closed.

Which of those two doors is better?  The one you first picked had only a 1% chance of being the car.  Most likely it's a goat.  What about the one that Monty left closed?  If your first pick was the car (1% chance), the other door is a goat and you shouldn't switch.  However, if your first pick was a goat (99% chance), the other door is the car and you should switch.  Most of the time you're better off switching.  Another way to look at it, it's kind of telling that Monty opened 98 doors and left that specific one closed.  Might that door have something special about it?  The one you first picked probably isn't special, since you were just picking at random.

I would support reorganizing HS math to include a good introduction to probability and statistics, since arguably those are more likely to be useful to the average person than algebra.  However, I think this only works if the class gets across basic concepts (including some of the fallacies K16 is reading about) rather than focusing on how to plug numbers into formulas for hypothesis testing and whatnot.

[smallcleanrat]

I'm not sure what the complex answer looks like, but I remember discussing this in a high school math class.

My understanding of the reasoning behind the answer (it is better for the contestant to switch his answer) is that the answer he chose has a 2/3 chance of being incorrect and only a 1/3 chance of being correct. The fact that the host eliminates one of the incorrect answers after the contestant has chosen a door does not change the fact that, when the contestant made his selection, there were two incorrect options and one correct option.

Imagine if the scenario involved five doors. Or ten. Or one hundred. In each case, imagine also there is still just one door hiding the prize. So all the doors except for one are incorrect choices. The contestant picks a random door, and then all the doors except for two are eliminated (the contestant's choice and one other door). Is it clearer why the contestant's first guess is more likely incorrect than correct?

EDIT: ah...Liquidambar was too quick for me

[smallcleanrat]

I also posed this problem to my dad (who has a master's degree in physics and certainly a higher level of maths education than I had in high school) and was condescendingly told I was confused because *obviously* the odds were fifty-fifty with only two doors left to pick from.

[kaysixteen]

Your dad is right, though-- regardless of what the overall math would be before any of the doors were opened, right now, there re only two doors left, one of which has a car, the other a goat.   So if Monte then gives the contestant the chance to change his answer, the contestant is left with a 50/50 chance of being right, whether he changes his answer or not.   I am no math genius, but I can see no other answer here.   The solution that smallcleanrat posed, however, is indeed the one Grimes and vos Savant argue, again with complicated stats, but near as I could glean, this would take into consideration all the possible answers before any doors had been opened.   Which IS different from when there are only two left.... unless I am missing something?

[smallcleanrat]

Your dad is right, though-- regardless of what the overall math would be before any of the doors were opened, right now, there re only two doors left, one of which has a car, the other a goat.   So if Monte then gives the contestant the chance to change his answer, the contestant is left with a 50/50 chance of being right, whether he changes his answer or not.   I am no math genius, but I can see no other answer here.   The solution that smallcleanrat posed, however, is indeed the one Grimes and vos Savant argue, again with complicated stats, but near as I could glean, this would take into consideration all the possible answers before any doors had been opened.   Which IS different from when there are only two left.... unless I am missing something?

I'm a little confused by this. Are you saying my answer also uses "complicated stats"?

Liquidambar and I both suggested considering the same problem, but with 100 doors (with only one hiding the prize). Would you say that the odds are still fifty-fifty in this case after the host eliminates all the doors except two?

The odds *before* the host narrows down the choice to two doors *do* matter, because the contestant made his initial selection *before* the choices were narrowed down. That's why the odds of his first choice being correct depend on the "initial math before any of the doors are opened."

[Parasaurolophus]

That's my understanding as well, but I lay claim to no real expertise.

But: I don't recognize this as an instance of the gambler's fallacy, which is about failing to recognize that statistically independent events are independent. Unless the point being made was that when it comes down to two doors, the odds are a coin toss and thinking they aren't is a failure to recognize that it's a new situation not dependent on the first? That makes it look more like a gambler's fallacy, but it also means either you misunderstood the point being made, or they got it wrong. Or maybe the game is you choose doors before any get eliminated?


It's worth pointing put, though, that everyone is bad at stats in everyday life. Our guts are constantly leading us astray. (Doctors, for example, are notoriously bad about the base rate fallacy--IIRC Stephen J. Gould has a good essay on the subject relating it to his cancer diagnosis. I think it's in Bully for Brontosaurus.

[smallcleanrat]

To say it another way, the question is really getting at whether the contestant's first choice (with all doors closed) was more likely to be correct or incorrect? If it's more likely to be incorrect, then he is better off switching when presented with the final two choices.

Since there are initially two wrong choices and only one right choice, he is more likely to have chosen wrong with his first guess. So it's better not to keep it. If he was wrong initially, the other final door is right (the host having eliminated all the other incorrect choices).

[smallcleanrat]

That's my understanding as well, but I lay claim to no real expertise.

But: I don't recognize this as an instance of the gambler's fallacy, which is about failing to recognize that statistically independent events are independent. Unless the point being made was that when it comes down to two doors, the odds are a coin toss and thinking they aren't is a failure to recognize that it's a new situation not dependent on the first? That makes it look more like a gambler's fallacy, but it also means either you misunderstood the point being made, or they got it wrong. Or maybe the game is you choose doors before any get eliminated?


It's worth pointing put, though, that everyone is bad at stats in everyday life. Our guts are constantly leading us astray. (Doctors, for example, are notoriously bad about the base rate fallacy--IIRC Stephen J. Gould has a good essay on the subject relating it to his cancer diagnosis. I think it's in Bully for Brontosaurus.

Yeah, this comes up a lot when I read cognitive science or behavioral economics. This type of reasoning is not very intuitive for most human brains.

I'm familiar with this example by the name of the Monte Hall problem. And it does include the condition that the contestant chooses *before* any of the doors are eliminated. The second condition with only two doors remaining does depend on what came before; the host eliminates doors such that there is one incorrect choice and one correct choice remaining. If contestant guessed right the first time the other door has a goat. If he guessed wrong, the other door is the car. So the presence of goat or car behind the other door depends on the contestant's initial selection (the other door has to be the opposite condition of the contestant's initial choice).

[Parasaurolophus]

Yeah, this comes up a lot when I read cognitive science or behavioral economics. This type of reasoning is not very intuitive for most human brains.

I'm familiar with this example by the name of the Monte Hall problem. And it does include the condition that the contestant chooses *before* any of the doors are eliminated. The second condition with only two doors remaining does depend on what came before; the host eliminates doors such that there is one incorrect choice and one correct choice remaining. If contestant guessed right the first time the other door has a goat. If he guessed wrong, the other door is the car. So the presence of goat or car behind the other door depends on the contestant's initial selection (the other door has to be the opposite condition of the contestant's initial choice).

Ah, yes, that makes sense. I think this may be what's tripping you up, then, kaysixteen: you (we!) haven't got the math wrong; you've (we've) got the game wrong.

[ergative]

The way I get my head around this is as follows:

Yes, the odds are 50/50 that the unselected door contains a car, but only if if the door that is revealed by Monte is chosen at random. But Monte is not choosing the door at random. He's choosing the door that has a goat, because he knows which it is. Consider the possibilities:

Door A = goat
Door B = goat
Door C = car

You choose A. Monte reveals B, which is goat. Switch = car
You choose B. Monte reveals A, which is goat. Switch = car
You choose C. whatever Monte reveals, switch = goat.

Monte is always going to reveal the door that hides a goat, which means the object concealed by the final door is not randomly selected from two options. Monte's knowledge in choosing what to reveal is going to ensure that what remains is not random. As a result, in two out of your three initial choices, switching will yield the car. Only in one out of the three initial choices, the choice when you chose the car in the first place, wwitching will give you a goat. So two thirds of the time you're better off switching.

[kaysixteen]

This is brain-buzzing.   I get the stats argument for what would be the case *before* any door had been opened, but when Monte opens one goat-containing door, and *then* asks the contestant if he wants to keep his choice or change it, it is self-evident that this *new choice* would be a 50/50 one, however different his potential choices would have been before any doors had been opened.... it seems, thus, that there are two different scenarios at play here, one before one of the doors (or 98 of the doors in a 100 door scenario alternative) had been opened, and the other when there are only two left, and one has to choose then.

The classic gambler's fallacy, which author Grimes also elucidates as being the case where you flip 20 coins in a row, all coming up heads, and then the fallacious gambler assumes that the next flip is more likely to be tails, because, presumably, of the law of averages, seems different from the Monte Hall problem above.   

That said, I still could be missing something, and I do plead ignorance of statistical mathematics.   I get that it would likely be better if kids were trained to learn stats.... but I am also more convinced than ever, based on reading several chapters' worth of statistical fallacies and errors, in Grimes' book, that statistics can be made to lie, very effectively, and be misinterpreted in  any case, by bad actors in the first place, and well-intentioned but facts- and/or math-challenged readers, in the second.

[smallcleanrat]

If there are one hundred doors and only one with a prize, with 99 wrong doors, the contestant is most likely to guess the wrong door. 99 ways to guess wrong; 1 way to guess right. The final question is whether or not he wants to stick with this first guess.

If his first guess was probably wrong, why shouldn't the contestant switch once the host eliminates the other doors? If the guess made when there were 100 doors to choose from is the wrong guess, it's still the wrong guess when the other doors are eliminated. Why stick with this probably wrong first guess?

The contestant is being given two choices at the end, yes, but that does not mean the odds of the first guess being correct are 50-50.

[ergative]

This is brain-buzzing.   I get the stats argument for what would be the case *before* any door had been opened, but when Monte opens one goat-containing door, and *then* asks the contestant if he wants to keep his choice or change it, it is self-evident that this *new choice* would be a 50/50 one, however different his potential choices would have been before any doors had been opened

No, this is where the fallacy lies. Nothing changes about the game after the door has been opened, because the choice of door that is opened is determined entirely by your initial selection. You have a 2/3 chance of choosing a goat the first time round, because there are two goats out of three doors. In that event, when you chose a goat to begin with, Monte reveals the other goat. There is no chance or probability about this. If you choose a goat, he will always show you the other goat. So in those two thirds of choices, whatever the remaining door is, it will have the car. Only in the 1/3 of cases where you chose a car to begin with will switching give you a goat. So switching is 2/3 likely to give you a car.

This is a small enough example you can also  just list the options: After Monte opens a door, which will always reveal one of the two goats, here are the possible scenarios. We'll label your first choice door as C, and label the two remaining doors A and B:

Scenario 1: A=goat/B=car;
Scenario 2: A=car/B=goat;
Scenario 3: A=goat/B=goat

In scenarios 1&2, you chose a goat to begin with. Monte then revealed the other goat (because remember, he always reveals a goat). So if you switch, you get the car.
In scenario 3, you chose a car to begin with. Both the other doors are goats, so whichever Monte reveals, you still get a goat if you switch.

But there are two possible outcomes where switching yields a car, and only one outcome where switching yields a goat. So you should always switch.

The classic gambler's fallacy, which author Grimes also elucidates as being the case where you flip 20 coins in a row, all coming up heads, and then the fallacious gambler assumes that the next flip is more likely to be tails, because, presumably, of the law of averages, seems different from the Monte Hall problem above.   

Yes, this is completely different.

That said, I still could be missing something, and I do plead ignorance of statistical mathematics.   I get that it would likely be better if kids were trained to learn stats.... but I am also more convinced than ever, based on reading several chapters' worth of statistical fallacies and errors, in Grimes' book, that statistics can be made to lie, very effectively, and be misinterpreted in  any case, by bad actors in the first place, and well-intentioned but facts- and/or math-challenged readers, in the second.

Very true--but I don't think you need statistical mathematics for this particular case. It's so small that you can just write down all the possible outcomes and get your probabilities just by counting up how many outcomes of the total give you a particular result. In general, though, I agree: people often have inbuilt assumptions about statistics that are just wrong*, and are therefore easily manipulated.

*A very common one: If there are X outcomes, then each outcome is equally probable. E.g., four possible outcomes, so each outcome has a 25% chance of happening. I usually use something like 'either an asteroid will strike tomorrow or it won't' to show the problems with that reasoning.

[Hibush]

This is usually framed in a sneaky way, which is what causes a lot of confusion. Monte Hall does not open a door at random. If he did, the odds would be 50:50.
Instead, he opens one of the two doors with the booby prize. That non-random act adds information.