Class: Informatics, Computing, and the Future
Instructor: Dan Berleant
Transcriber: Brooke Yu
Date: Tuesday, February 05, 2013
Professor: Did anybody no get a copy? so take a look. It's on the Delphi method. You have the data. You just need to process it.
Professor: Alright, so any questions on this homework? What do you do or something?
Male Student: Do we just chart out our readings from our question?
Professor: Yeah. You'll end up with a graph that will probably look something like this. This is the median.... This is the 75% mark, 25% mark. You could put a bar or something to indicate the low and high estimates
This is the middle 50%. Okay?
You know, this sort of presumes you can divide the data into 4 equal sized sets of data, right? But unless the amount of data points you had is not divisible by 4, you'll have to adjust, so some of these will be between two data points
If you have a in even number of data, then the median is the between the two middle numbers. You can kind of figure out how to do that.
Question number 4 gets back to the term project, but also to sort of what we want to get out of this course. It guides you to think about the problems the course wants you to look at.
Any other questions?
Alright. So it'll be due in a week. And we're actually done covering the Delphi method, so we'll go onto something else today. By the way, does anyone not have their Delphi method data? Did you all get yours? Anybody not have their data yet? Great. I should point out that this only has to be done on the second iteration. You don't have to do it twice.
Last time we looked at prediction markets, which is another way to predict the future. Today we'll look at it in a little more detail- this concept of prediction markets.
Okay? So, you know, the Delphi method was one way of getting crowd source predictions. The problem with that is if you're wrong, it doesn't matter. You think if people sort of have to put money on what they're guessing or estimating, then they'll do a more careful job of it, and that's what prediction markets are about. It's about buying and selling predictions so people will try harder to get it right.
So that's what prediction markets are all about. The word market, by analogy, is like the stock market.
So people make predictions all the time. They don't always do it in prediction market, but people are always making predictions. I guess what I want to do with point number 2 is establish that making predictions is sort of an intrinsic human activity that could help explain why we are the way we are.
Like, it could explain why we're the smartest species on the planet because of the need to make predictions.
Let's look at the kinds of predictions people make. Everyone makes them. So predictions people make and are better than animals at.
Well, people think about people's reactions, which is a prediction. It's a very human activity to do.
Okay? The ability to predict the effect of your act on others is useful. If you can predict how people will react, you can gauge your actions accordingly
In a highly social species like humans, it's important to figure out how people will react so you can work and operate effectively within a social context.
In order to do that, in order to really predict reactions, you need a theory of mind, which means you need a model in your mind of other people's minds. When you think "how will they React" is sort of simulating their mind in your mind.
A dog might be able to figure out how you're going to react to some degree, but they don't do a very good job of this. They don't simulate other minds in their mind, which is what we do all the time
You know, when I'm making up a homework, I think "well, if I state this problem this way, how are the students go to interpret it? Will they understand it? Alright?" At the same time, you as student wonder when you're doing an assignment "is he a tough grader? An easy grader? Do they really want That?"
This is why people sort of don't know how to do assignments that well until they learn what the professor really wants. Or tests, you know? I don't give tests very often, but it's the same deal. Unless it's multiple choice, you might wonder if the professor will grade higher if you use certain words.
So people have the capacity to have a theory of mind where they can model other people's minds in their own minds. This may not sound like prediction markets, but we'll get there.
Basically, when you're a member of a social group- like dolphins too- they seem to do this sort of thing too. You're in a game theory scenario. This is where two parties are adversarial. One will win and the other won't. In real life the person with the better theory of mind ends up ahead of the game. There's a whole academic field called game theory where you analyze and try to understand how things work in competitive situations.
Well, turns out that this need to have a theory of mind and to compete in a social setting is likely to be one of the causes of human brain size. Humans have pretty large brains. Well, not that big. This is 2 feet wide. Humans have, not the biggest brains on the planet, but among the biggest.
You know, look at the ratio of body size to brain size, humans are basically the biggest.
The concept here is that populations in a social setting tend to compete in various ways, and that causes bigger brains to develop or to be selected for.
This is an outline of a human brain.
This, on the other hand, is a chimp brain. They're pretty big, but not nearly as big as a human's.
They have that wrinkling too, but it's greater in humans. We have more wrinkles on our brains, showing we have more surface area folded to fit into the skull.
Chimps are smart, but not as smart as humans.
Okay? So this is called the social competition theory of the development of the human brain
So there are different theories about why the human brain is big. Many researchers favor this idea though.
We're never going to prove absolutely because we don't have time machines, but we can think about it.
Okay, so anyway, where I'm going is that this need to predict and the difficulty of predicting explains intelligence.
So what is it we need to predict?
So somebody- what is it we need to predict? If our need to predict explains why we're so smart, what is it that we need to predict?
Male Student: Weather.
Professor: Okay, if you're a farmer and in ancient times when you could starve if the crops failed, you might need to predict weather. What else might we need to predict? I think we can generate a couple more before we go on. Anything?
What did I mention up around the ceiling?
Something we need to predict....
Okay, how about I give you a hint? Social competition. Is that enough of a hint? What do we need to predict?
Remember I said something about having a theory of mind? And they think some animals have it? Why do we need a theory of mind? What does it help us predict?
So what you can understand what other people are going to do. You have an internal theory of how people's minds are working. How could that help you?
Male Student: So you could predict if another country is going to try and kill you
Professor: Right or if the caveman in the other cave is going to hit you with a rock.
Professor: Or, you know, if the tribe only has ten rocks available and 11 people, you don't want to be the person that doesn't get a rock.
So the theory goes that humans are humans because we need to predict in order to compete with other humans in a social setting.
You know, elephants are pretty smart and they travel in groups. Porpoises travel in groups too, as well as chimps. Deer- well, deer can travel in groups, but they're not very smart.
But of course, in addition to the social competition theory of the human brain, other predictions are important. Like the weather. Someone mentioned that. The country puts a lot of money into making highly sophisticated weather forecasts.
This is a picture from Portugal. It's a picture of a weather pattern. Sun is setting, lights are on the in the village. I don't know what's going to happen, but it looks like rain.
Here's something. A waterspout is a tornado over the water. A tornado can draw water up into the funnel. You can see the spiral on the surface of the water that's how much wind is down there. You know, water's very heavy. It doesn't take that long for a waterspout to lose its energy. It's amazing it can do it a all.
Indeed, the national weather service is a large government agency that is computer oriented- they have a lot of computers used to predict the weather scientifically.
A lot of money goes into predicting the weather because it's so important economically. We know the weather man isn't perfect, but it's a lot better than it was. Now they have predictions that go up to 10 days all because of computers.
So weather prediction uses computers. The Delphi method doesn't. I think I'd be better off going to weather.gov than asking you guys what the weather will be.
I'm sure weather.com uses government information to post that.
There may be companies that do it too, but not your local TV station.
Okay, so there's other forms of group wisdom besides the Delphi method. For example, sports betting. People like to do that often.
And, you know, you should be able to determine what the odds are that a team is going to win a game based on large numbers of people betting. In fact, groups that kind of serve as a clearing house for sports bets will issue odds that they'll sell bets for, and sometimes those are picked up by the media.
The reason that's even news at all is because it's an estimate of the actual probability. That's just the kind of- that's exactly what a prediction market is except it has a few different wrinkles. It's getting a bunch of people to make bets on what they think will happen and they use money so they'll get good estimates.
Here's another example. This is from last year's class.
So last year in this class, I asked "will the average global temperature for 2012 be the highest ever recorded." There was a company selling those predictions.
Remember, there were predictions- if it turned out to be the highest ever recorded, you'd get $10. If you said no, you'd get $0. If you thought it was going to happen, you'd buy. If you were even more confident, you'd buy a prediction for $9. If it didn't happen, you'd lose $10. You'd have to be sure to buy such a prediction.
So it turns out last year at this time, you could have bought a yes vote for $2.50. Ultimately, you would have lost your $250.
Turns out 2012 was the 9th hottest year on record. So it was hot, but not the hottest, so you would have lost your $2.50.
Per share, right? If you bought 10 shares, you would have lost $25.
This assessment came out just a couple of weeks ago.
[Teacher reading: [On board.]
Professor: This is a pretty cool map. It shows the difference between 2012's temperatures and the average temperatures at various locations around the globe. You can see it's colder down here in Antarctica. You can see the farther north you get, the greater the anomaly.
It's a lot warmer than one would have expected on average.
Okay. So why am I going on about last year instead of this year? Well, because the company that offered those bets where people could buy those predictions for $2.50. That company has problems. The company was called intrade.com.
Turns out the company was basically barred from business in the US.
It was done in November- not long ago.
Professor: Basically, they forced intrade to not do business in the US. A few months later it turns out the intrade is still in existence, but it's a much smaller entity. This morning when I checked to see if you could buy or sell predictions on the same question for 2013, you couldn't.
So it's a much smaller operation because most of its business was in the US, even though it's an Irish company.
Anyway, the person writing this article was pretty upset about it, not because they wanted to make bets, but because it's... let's see.
Well, first he claims it's harmless, but, you know, the US already allows other forms of gambling.
The problem is that prediction markets work. They give good information about the real probabilities about future events. And so by preventing people from using them, they're removing a tool that people can use to predict the future.
You could look at the prediction market and see the odds to figure out a good estimate. We can't do that anymore in the US with intrade. They lost most of their customers.
So according to this author, by making prediction markets illegal, [Teacher reading: [On board.]
So this person who wrote this article was very much against this. They think prediction markets should be allowed.
So the company did not go out of business. We'll look at it in a minute.
Fewer predictions that you can buy and sell, fewer people making predictions. I didn't try one myself, but I'm sure if I did it'd say I had an IP address in the US.
Male Student: You could just change your IP address.
Professor: Yeah, but the feds might come knocking on my door. The company had its share of bad luck. The founder of the company died at 42 before this happened. He died.... climbing mount Everest.
He was within 50 yards of the summit of Mt. Everest. He was clearly a risk taker and he started a company then tried to climb mount Everest. That's very dangerous. If you want to try, it's really expensive to do, and a significant fraction of people who try don't make it.
Meanwhile, he has a child who will grow up knowing his dad tried Mt. Everest and didn't make it. Okay, let's see what all the excitement is about and to go intrade to see what it's like.
If you look at the home page, it looks like it always did, but it has fewer predictions you can buy and sell. Let's look at some.
Markets- Here are scientific markets. They used. Have dozens, but now there's only 5.
[Teacher reading: [On board.]
Here are some related to ET life.
Here's one about physics. There's some activity- people are betting on these, but it's not nearly as many as there used to be. There used to be several dozens of scientific things.
Politics- well, since Americans can't use this site anymore, there's less of a focus on American politics. There's some, but Italian politics rates pretty high on this list, but I don't follow Italy's politics.
Construction and engineering. Three of them.
Will the world trade center be re-opened by the last day of December 2013. That's looking pretty good right now. So you could buy a prediction for $8, and if it happens, you'd get $10.
I was looking at this earlier. A lot of these markets have few people buying and selling, so the prices are pretty erratic.
Okay. We can look at a few more in detail if we have time at the end of class.
Okay, so at intrade, every prediction is worth $10 or $0. Why do you think a prediction contract or the price of a prediction will tend to get closer to $10 or $0 as the prediction gets closer to happening?
Why do you think?
Male Student: Because there's a certainty of it happening or not happening?
Professor: Right, so things become more certain the cosier it gets. So if you're betting on whether the year will be the hottest year on record, you don't know much in January, but you know more in December. So the price of the contract will get closer to $10 or $0 if the evidence shows it's likely or not.
How about this one. Could the price of a prediction get high and then end up low or vice versa?
Male Student: I think if they came out with new support for a prediction, you know, for or against it, it could skyrocket up or fall down
Professor: You know, let's suppose you could buy a prediction for $1. That means, people only thing there's a 10% chance. If it seems more likely, the price will go up.
I followed intrade in the last election. That happened a lot, you know? When people thought one candidate was ahead, the price of the prediction they would win would be high. Then it might drop as the ratings dropped.
So yeah, could a price get high and end up low? Yes.
Sports have always been something people wanted to bet on.
You know, what's the difference between sports betting and buying a prediction? Nothing. It's the same kind of thing.
It's hard to throw a ball up that high. I'm not a big tennis fan, but my wife is, so I see some of it.
So sports betting has a long history because people want to predict games and win bets.
You can imagine the ideal bookmakers. An honest bookmaker will offer odds to a sporting event that results in equal payoff for the bookmaker, regardless of outcome. So the bookmaker would set the odds they're willing to buy and sell debts such that the number of people who bet for one team vs. the other, when the bookmaker takes the money from the people who got it wrong and gives it to the people who got it right, they'll end up even.
How do real bookmakers differ from that?
Male Student: It's not equal.
Professor: Right. Does anyone know how to a roulette wheel works? How does the house get a cut of the roulette?
Male Student: I've never bet in my life.
Professor: How many different outcomes are there? Like 32? Then there's an extra one for the house. So there are 33 outcomes instead of 32, but if it lands on a 5 and you bet 5, you get 32 times what you bet.
So Vegas is the same way. They skew the payoffs and so on so they get money and make a profit.
Or if you had a system that was fair in that sense, then the true odds according to the group wisdom would come out the odds that would result.
Okay? Here's another example- political predictions. Election season- media, ordinary people, etc. predict what's going to happen. Remember the last election- it was nonstop on the news
Polling, trend analysis, analyzing demographics, etc.
Everyone has a different opinion, and that's why you could imagine instead of asking individuals using groups.
Polls aren't bad. They often work, but they don't work well in the last election. Prediction markets have been claimed to do better because you're not just asking people on the phone. You're getting people to put money on who they think is going to win and people are more careful with their judgment.
Predictions markets are perfect on this front either. Let's look at the last election.
Okay. This is going back to November. Here it is.
Okay, so this is from last November. The person who wrote this blog was kind of famous at the election for predicting everything so well. Here, he's talking about how predictions markets did. He's talking about intrade because it was legal back then. So a lot of people had good data.
In the last election, it was a legal company, and it turns out that in intrade, the trades for romney were more favorable than in other markets.
So there's some questions about why intrade or why one prediction company would- the crowd source wisdom there would differ from other places.
Romney was given a 30% by intrade, but a 15% chance by another company.
So why would that be? Why would one prediction market get another result from another?
Here are some possibilities. How can these be skewed?
Male Student: You could take down the server.
Professor: Okay, can anyone think of a non-computer related way to make it give the wrong answers in essence?
Well, let's suppose you decide "I don't care whether I make money or not. I just want to make the price of the prediction go up." Then you could just go and buy lots of predictions. You don't care if you make money. It's like stocks- you drive the price up. If you drive the price up, maybe the media will talk about it.
Maybe they'll think the candidate will be more likely to win, so maybe you do that to get publicity for your candidate.
I you do it for reasons other than trying to maximize your financial position, even for other reasons, then you'll distort the predictions because the prices will change in response to what you're doing, but you're not trying to make money. So in that sense, you're manipulating the prices. Another thing that can skew results is if people buy predictions, not because they're trying to manipulate, but for other reasons.
Let's suppose there are two sports teams, and you really want.... we'll call them team cool and team uncool. You really want team cool to win. You'd think you'd bet on team cool, right?
That might not be the right thing to do. Perhaps the right thing to do would be to bet on team uncool, because then if team cool wins, then you're happy to lose $5. If they lose, you'll make $5, and you'll only be a little sad they lost.
So that's one reason why you bet against the team you want to win. If they win, you're happy. If they lose, you make money but you're not too unhappy.
So that's not irrational at all. You might just want to try to control the amount of drama in your life. You don't want to be too upset if they lose. Or maybe you want more drama and so you bet on your favorite team. If they win, you're happy and you make money. But if they lose, you're sad and you lose money.
The minimum drama approach would be to bet the amount of money a win is worth to you. Then there's no drama at all. You don't really care.
But in any case, people can do that and there's no reason why they wouldn't do it in an election.
So it's irrational in the sense that it's not specifically geared to making money. It's buying bets for another reason, so it skews the price of the predictions, and you get bad predictions.
These are reasons why putting money on your bets is not always a perfect way to get the crowd source wisdom.
Any comments? Thoughts? Questions?
Okay, well, buying and selling predictions is kind of like the stock market. Economists look at the stock market because it's a leading economic indicator. Who knows what that it? The opposite of leading and lagging.
Male Student: What shows the future, kind of
Professor: Exactly. Leading refers to showing the future. So economists like leading indicators because some obscure price of something that they feel is a leading indicator goes up, that means they think the economy will go up.
That's why they like leading indicators. The stock market can be a leading indicator, so the stock market is kind of a prediction market. People buy stocks today because they think the stocks will go up more tomorrow or next year, and normally they go up if the company is doing better. So people buy stocks because they think the company will do better and make more money next year.
So the stock market is something of a leading indicator.
Okay. Well, you know, prediction markets are kind of... an abstract concept. If you were to imagine a way to use group- to collect wisdom from a group, you know, you might come up with a Delphi method. You ask people and get an average. That sort of makes sense.
But you probably wouldn't come up with a prediction market, because it's a little abstract. It's clever and hard to dream up.
So you probably wouldn't come up with a prediction market, so that's why it's sort of a modern concept.
The earliest design for prediction markets appeared in a science fiction story from 1975. He called it a Delphi Pool. It was related to the Delphi method but he added money to it.
Making people use money makes people bet more carefully, and presumably better. Here's an example of how this was applied.
This happened a few years ago. There was an attempt to make a prediction market for terrorist attacks.
So why have a prediction for that? Why buy and sell these predictions?
Can anyone imagine why they'd want to have a prediction market for that?
Male Student: Like they predicted if it would happen?
Professor: Yeah, people would make predictions on that
Male Student: People bet on anything.
Male Student: And if you bet then do it, you win.
Professor: Exactly. That's exactly what people were hoping. They thought that if terrorists wanted to make money, then the few people who knew about the attack would go and bet that it was going to happen so they could make money. The people who designed this prediction market would hope that if there was a spike in a prediction, that'd be an indicator something would happen.
Male Student: That's just kind of creating something you're trying to stop.
Professor: That's right. So it's sort of an incentive to terrorists to do it. So you might think "what the heck is the US government Doing?" Seems like a crazy idea, doesn't it? Does it sound like a crazy idea?
Male Student: I don't know. Might be a government cover up for 9/11.
Male Student: Did it pay out or anything?
Professor: Well, here's what happened. They set this up a number of years ago. It was called PAM, but I forgot what that was- policy analysis market. It wasn't just for terrorist action they just wanted to sneak those in there
Anyway, a couple of senators found out about it, so they canceled the program, and a program director at DARPA had to resign.
When these things get politicized, you can imagine how politicians act
Somebody has a role, so the program director who permitted it to be set up had to resign. To answer your question, from what I could see, it's not clear if any terrorism predictions were ever actually traded.
But it's a clever idea, but it does seem to encourage ideas.
Male Student: I would feel bad if I bet on a town getting attacked and then it got attacked.
Male Student: You'd feel a little convicted about it.
Professor: Let's see where we are. We can look at some more predictions if we have time. I'm in a bit of a quandary. I had homework before for people to go on intrade and to pretend they had $100 and see if they would make money pretend on their predictions, but now. I mean, I may still do that, but intrade just doesn't have as much available, and I can't ask you to put money on it because it's illegal now.
Intrade wouldn't even take your bets unless you changed your IP address.
Male Student: Make a paypal account from another country.
Male Student: So the whole thing's not legal?
Professor: Well, I guess it's legal in Europe.
Male Student: I just thought it was certain type of predictions.
Professor: I don't know the details. I think it was always in the illegal grey area because gambling. Anyone know anything about the legality of online gambling?
Male Student: I don't really have money to gamble away, so no I don't.
Male Student: I know there's always a lot of places popping up that do online gambling illegally. If they just put them in inconspicuous places and get caught, they just move them. They do that a lot where I'm from.
Professor: The whole thing is pretty shady.
Professor: Anyway, intrade used to be legit. I don't know about that other site. I don't know if we should try to do this or not. Mostly, the thing is that they don't have as great a selection of bets as they used to. What do you think. What to try it? They have some bets- excuse me, predictions.
Or should I try to redesign the homework?
Male Student: Maybe something a little less sketchy.
Professor: Well, it's not illegal to pretend.
Professor: Well, what do you think? How many people want to do it? How many people don't want to do it?
Male Student: I'll do it.
Male Student: I'll do it.
Professor: The concept of a prediction market is a good one. It is a way to predict, and as that one writer said, it's unfortunate intrade is not allowed to operate in the US because it reduces our ability to tell the future and you can make better decisions if you know the future. I wanted to try one more thing. We spent some time doing the Delphi method predictions. I thought we could try the Delphi method and then try to pretend we're putting money it. Let's just do it.
So here is a question. Will a live person land on Mars by 2024? Okay?
This is a little different from the Delphi method. We're just asking yes or no. Not a date. So any comments about why you vote one way or another on this?
Anyone remember what the Mars one organization is?
Female Student: Aren't they supposed to go in 2023?
Male Student: Didn't they shut down the whole NASA thing?
Professor: Well, NASA hasn't sent anyone out since the 1960s.
Male Student: I thought Obama cut their funding.
Male Student: They ended the space shuttle program.
Professor: There's an international space station, but I don't know what the status on that is. In my life, it's always been an argument. People have always argued over the cost of the space program and feeding poor people or something. It's an argument that's not going to go away. Going to Mars is a cultural activity and it costs a lot of money and people have to want to do it. When budget crunches happen, those kinds of questions come up. So there's nothing new about it. That argument has been around for a long time.
So they're always cutting the budget then they add to it, then cut it again.
So any other comments about whether or not a live human will land on mars by 2024? Think to yourself right now. Don't tell anybody just yet. We'll have a vote. I'll ask for all the yes's and all the no's.
Alright, how any vote yes?
How many people vote no?
In the delphi method, we'd talk about it. But now we're going to do something different. I'm going to ask you to think deeply about ... let's see. How can I state this?
If you're right, you're going to get $5,000. You have to wait a few years, but you'll get $5,000 if you're right.
And if you're wrong, you get $0
So think really carefully now. Think about this. And then we'll revote to see if anyone changes their mind. Now, pretend it'll really happen. I'll give you a minute to think about that.
Think about that $5,000
Okay. Let's try again. How many people vote yes?
How many people vote no?
Okay, so when $5,000 are at stake, people do it differently. You can see why the delphi method gives a different estimate than a prediction market. People say prediction markets are better because you care more about your answer.
Okay. We have 5 minutes left, and I think we should look at a couple of other sites.
By the way, UK football means soccer.
Casino, poker, horse racing. Let's take a look at sports in general.
Well, American football. We can do that.
Should have done this before the last super bowl.
So this is.... oh, do these countries play American style football?
Male Student: I'm going to say no.
Professor: Anyway, it looks to me like betfair is a sports betting company. It's a prediction market for sports.
Nope. That's just an explanation.
Looks like intrade and betfair are the two big ones. Let's look at intrade one more time and then we'll adjourn. Anyone want to pick one of these?
Male Student: Politics.
Professor: Let's see what's going on there. Anyone into Italian politics? Australian?
Male Student: I'm not into American politics.
Professor: Alright, let's see about Italian politics. Which party will win 2016. Oh, here's some US stuff.
Okay, so will democratic party retain the US Senate in 2014? It looks like the current.... I guess this is 0 to 100, so the current estimate is at 63% that democrats would retain the Senate. So it's a 2/3 change the democrats will retain the Senate.
The house of representatives in 2014, they have an 87% chance of that. You can buy a bet- actually, you can't, but it would be $8.30
Male Student: You could go to hidemyass.com
Professor: Haha, okay. I won't require you to use it in homework 4. Alright, so anyway, that's prediction markets.
Think about them next time you bet on sports and we'll go into something else nex time.