Hi, welcome to lesson five of Critical Thinking for the Information Age. We'll be talking about prediction today, how to predict what will happen in the world more efficiently than we sometimes do. We'll start the lesson with a quiz. Then we'll go back through the items on the quiz and see how you did, and whether it would be possible to improve on your performance. There is a phenomenon called Rookie of the Year, jinx. In American baseball, the Rookie of the Year, he's the best player for the year, is only rarely the best player the next year. The phenomenon is called the sophomore slump. Why do you suppose it occurs? Please say your answer out loud, or better still, write it down. Second problem is about my friend, Catherine, who's a foodie. She enjoys eating in fancy restaurants, but she finds that when she has a really excellent meal in a restaurant, subsequent meals are rarely as good. Why do you suppose that is? Please say your answer out loud, or better still write it down. One more quiz item, it's based on a true story. A psychologist was once lecturing a group of flight instructors. He told them that psychologists have found that skill training goes better when you give rewards for improved performance. That's more effective than giving punishment for mistakes. One of the instructors immediately raised his hand and said, lots of times I've praised flight cadets for clean execution of some aerobatic maneuver and the next time they try the same maneuver, they usually do worse. On the other hand, I've often screamed into a cadet's earphone for bad execution, and in general he does better on his next try. So please don't tell us that reward works and punishment doesn't, because the opposite is the case. So are you inclined to believe the psychologist or the flight instructor? Let's go back to the sophomore slump. The baseball Rookie of the Year is only rarely the best player next year. University of Michigan students who have no statistics always give causal interpretations of this. They say, well, maybe the pitchers make the necessary adjustments, or maybe the guy gets too cocky. But let's go back to our old friend, the normal distribution. How did the Rookie of the Year become the Rookie of the Year? Well, first of all, by having more talent, for sure, than the average person, a lot more, but everything went his way that year. He's way off here at one end of the continuum. But in that space in there, the really very top space, there is lots that can go right or can go wrong and that can affect your ultimate performance. Everything went his way that first year for Rookie of the Year. He got ideal coaching. He happened to do extremely well his first few games, built his confidence. He got engaged to the girl of his dreams. But the next year, he got an elbow injury, and I'm sorry to say, his fiance broke up with him. So somebody else got all the breaks. Somebody else was way out there on the end of the continuum. Being Rookie of the Year is an observation. It's an observation based on a very large number of behaviors, but it's still an observation. And like all observations, it needs to be thought of as being composed of true score plus error. Or to put it in a way that makes more sense in this case, the observation best that year is true score plus luck. So the Rookie of the Year had a lot of things that boosted his performance that would constitute an error with respect to what his true score is, or good luck. The next year, the luck didn't contain such an imbalance. His luck dealt him a new hand. Some other super-talented player got all the breaks. The error was on their side. The concept here is the concept of statistical regression. Extreme events of a type that is distributed normally will be followed and preceded by less extreme events, to the extent that the events are subject to chance influences. To help think about the Rookie of the Year problem, suppose we looked at the best players the second year. Would they likely have been the best player the first year? No, because regression works in both directions. And it would be silly to look at the best player the second year and find out that he wasn't quite so good the first year and declare that there must be a freshman slump. There's no slumps here, there's no jinxes, so far as we know, when we certainly know there's regression going on. So let's return to my friend Katherine who's often disappointed when she has a second meal at a restaurant where the first meal was excellent. Most University of Michigan students with no statistics will say, again, they're trying to make a causal interpretation. They'll say, well, maybe the chefs change a lot or maybe her expectations just got too high and the meal couldn't meet those. Well, meal quality is a variable, and excellent is an extreme value on that variable. And you have to expect that the next meal will be less extreme. To help you see that, think about whether you believe there are more restaurants in the world where you always get excellent meals, or more restaurants where you sometimes get excellent meals. If it's sometimes, then if your first meal at the restaurant was excellent, odds are that's one of the restaurants where you only get good, excellent meals sometimes, then your expectation for the next meal is that it's going to be at least slightly less good than the first. And about that pilot training. An extremely good maneuver is going to be followed on average by something less good. And an extremely poor maneuver is going to be followed by something less bad. That's what regression is all about. So it's the psychologist who's right. The flight instructor failed to recognize that the phenomenon of regression creates an illusion of causality. Anything that happens before you get a less extreme event after an extreme one is a candidate for phantom causality. Incidentally, the regression principle tells us that we give too much credit to doctors. Most diseases are self-limiting. They go away by themselves. But we're likely to credit the cure to the doctor's treatment. The doctor can be pretty sure that even if the treatment is a sugar pill placebo, the patient is going to get better and thank the doctor. Regression also helps to keep all kinds of nostrums and homeopathic remedies popular. I had the flu and I took snake oil and it cleared it right out. I went through a period when I kept getting headaches and I started putting wolfsbane on my cereal and they went away. Notice there's a bit of a man who's statistic quality to some of these things as well. Now let's talk about a slightly different kind of problem. Namely cases where there is more than one variable that you're trying to think about or more than one object or person. Again we'll start with a quiz. Highly intelligent women tend to marry men who are less intelligent than they are. Is that suprising to you? Why do you suppose it's the case? Please say your answer out loud or better still, write it down. Suppose I told you that the correlation between the intelligence scores of spouses is less than perfect, less than 1.0. Would that be surprising to you? There are number of tests of sense of humor that have been created by psychologists. Your lecturer, that's me, has a student who took one of these tests and scored at the 98th percentile, indicating that he had an extremely good sense of humor. What do you suppose is his percentile score on a test of IQ that he took at about the same time? Again, please say your answer out loud, or better still, write it down. What do you suppose is the correlation between sense of humor and IQ? Does your answer make you inclined to change your prediction about the student's IQ score? Based on the regression principle, you now understand that extreme scores on a particular variable don't directly predict future scores on that variable. This also goes for predicting one variable from another. Usually, the correlation between variable A and variable B isn't 1.0, but we treat it that way sometimes. People are sometimes surprised that extremely tall people have children who are considerably shorter, but they shouldn't be. The regression principle holds when we're talking about two variables, just as it does when we're talking about one variable. And the regression principle holds when we're talking about more than one object or person, father and son, for example. If a man is 6'4", what's the best guess as to how tall his son will be? You know almost enough to figure that out. Let me remind you that the mean for male heights in the US Is 5'10". The standard deviation is 3, so 6'4" is 2 standard deviations from the mean. And recall that the correlation between heights of fathers and sons is 0.50, or actually, of course, what I told about is the correlation between the heights of mothers and daughters. But it's the same for fathers and sons. So you only need to know one more thing to know what is your best prediction for the height of that son of the very tall father. We know we have to go halfway from the mean, that's 0.5 of the mean, to get the prediction for the son's height. That's 1 standard deviation and the height at 1 standard deviation is 3 inches above the mean which is 6'1". So here's how to estimate one value from another which is more extreme. Find or estimate the correlation between the two types of events. Go that far away from the mean in the direction of the extreme value to get an estimate of the value you're trying to predict. So if the correlation is 0.5, you go halfway from the mean to the more extreme score. If the correlation is 0.3, you go 0.3 of the way from the mean to the more extreme score. So let's go back to the fact that highly intelligent women tend to be married to men who are less intelligent than they are. Most people find that surprising, and once they know it, they immediately start looking for causal interpretations. Well, maybe some women who are really super bright, maybe they don't really want to have the light shining on somebody else all that much. I've heard a dozen different causal interpretations. But did you find it surprising that the correlation between spouse's IQ was less than perfect? Probably not. It's too easy to think of exceptions to that. And in fact, the exceptions are the rule, because the correlation between spouse's IQs is only a little more than 0.3, let's say, 0.33 for easy of calculation. So if you know the particular woman has an IQ of 130 which is two standard deviations from the mean, that's 100 plus 15 plus 15. What is the expected value of her husband's IQ? Well, back to the normal distribution. The metric we use to measure IQ is defined in standard deviation terms. So we go one-third of the way from the mean to 2 standard deviations above the mean, that's 130, so we go to 110. That's 100 plus one-third of the difference between 100 and 130. Notice that imperfect correlation means regression is a certainty. If the correlation between two variables is less than 1.0, which it is for almost everything that we can apply statistics to, an extreme score for one variable will be less extreme on the correlated variable. But bear in mind the regression principle can only tell you your best expectation. After all, some truly brilliant women are married to men who are even smarter. But most truly brilliant women are married to men who are merely bright. And most truly brilliant men are married to women who are merely bright. So let's go back to that test of sense of humor. This student scored the 98% percentile on it. And I asked you what his percentile score would be on an IQ test. An awful lot of people say the 98th percentile. But of course, that could only be true if the correlation between sense of humor and IQ were 1.0, literally perfect. The person with the highest score on sense of humor has the highest score on IQ. The next highest score on sense of humor the next highest IQ. Nobody believes that. So expected IQ couldn't possibly be 98th percentile, it has to be less than that. And to come up with a good guess as to what it should be, estimate what you think the correlation is. What would you think it would be? I think it's probably no more than about 0.5. So I should go 0.5 of the way from the mean, which is 50th percentile to the 98th percentile, which is one standard deviation away from the mean which is the 84th percentile. That's my best prediction. So to summarize this, extreme values will likely be followed by or associated with less extreme values on average. The more extreme the value, the more likely it is that other values on the same or different dimension will be less extreme. The lower the correlation between dimensions, the greater the regression from one dimension to the other. Note that if the correlation between two variables is zero, you have to predict from even the most extreme value on one variable that the value on the other variable is at the mean. So if the correlation between the number of movies that people go to and their height is zero, even if a man goes to 500 movies a year, his expected height is 5'10". In the next segment, we're going to talk about another non-obvious statistical concept that's necessary for all kinds of prediction purposes, namely base rate.
