614: Making Smarter Decisions When You Can’t Know Everything with Annie Duke

Pete Mockaitis
Annie, welcome back to the How to be Awesome at Your Job podcast.

Annie Duke
I’m excited to be back. It’s been a while.

Pete Mockaitis
It has. Well, yeah, just looking at that, it’s been over two years. Wow, time is flying, because I still remember many of the things you said kind of closely, like, “Want to make a bet?”

Annie Duke
Nice.

Pete Mockaitis
And so, it seemed closer. So, yeah, I’m excited to dig into some wisdom you’ve formulated in your latest book How to Decide. But, first, I think we need to hear, we know about you being a poker champion, but I just recently learned that you’re also a Rock Paper Scissors Champion and I want to hear the whole story.

Annie Duke
Oh, my gosh. There’s, like, literally so little story to this. It sounds much more amazing and glamorous than it actually is. At the World Series of Poker one year, some friends of mine, like, they organized a Rock Paper Scissors World Championship which was designed like March Madness. And I quickly went over and asked my friend for some rock paper scissors advice, which he gave me, and I ended up winning.

Pete Mockaitis
Well, that’s good advice. So, what’s the trick?

Annie Duke
Well, first of all, a lot of luck. Well, the trick that he told me and, listen, I’m not certifying this advice, it happened to have worked for me, is that you should be thinking about how you can tie with the person. So, it’s a little bit like anything else that you’re playing that’s like that. You want to try to get into the other person’s head and think about what they might be throwing. So, if they’re throwing scissors, you should be trying to throw scissors back.

Pete Mockaitis
Okay. And so…

Annie Duke
And I think the reason for that is that if they were thinking about you being able to predict them, which is where people’s heads go, so if I’m thinking about throwing scissors, I’m worried about you throwing rock. So, if I changed my mind, I’m going to go to paper, but scissors beat paper. So, I think that’s what it is. It’s sort of you’re going those levels deep, that “The person is thinking I’m throwing scissors but what if they know?”

Pete Mockaitis
Yeah.

Annie Duke
“And if they know, then I want to figure out something that’s going to beat that.” And so, when you’re shifting off of your original intention, you’ll lose to the tie.

Pete Mockaitis
Oh, man, there are so many layers here.

Pete Mockaitis
I’ve looked a little bit in the world of championship rock paper scissors play, and I understand some people will just pre-memorize a script, like, “I’m going to go rock and then scissors,” and then just roll with it regardless of what you’re doing.

Annie Duke
Yeah. So, I’ve used that strategy before. So, basically, what you’re saying there is, “I don’t want to be predictable,” so you would do this if you thought that your opponent was actually quite good. In other words, so you felt like you couldn’t predict your opponent then you would want to go to, essentially, a random number generator. So, that’s basically what they’re doing. They just write down a script in advance, and they’re just saying, “If I’m not reacting to what they’re doing or reacting, whatever, then you can’t predict me.” So, the way that I did that, there was one…I don’t know if it was in that tournament, it might have been another one. I took out a dollar bill.

Pete Mockaitis
So, there’s multiple Rock Paper Scissors tournaments under your belt.

Annie Duke
Two. So, what I did, I think I came against somebody who I thought was actually quite good at rock paper scissors, and so I took out a bill. I just had like a stack of bills, like dollar bills, and basically that would give me a serial number, it’s like 10 numbers or something. That would give me 10 throws. So, I had like, if it was zero, one or two, I would throw rock. If it was three, four, five, I would throw scissors. And if it was six, seven, eight, I would throw paper. And then I ignored nine and moved one. So, it was that kind of thing, so that ends up accomplishing the exact same thing.

Pete Mockaitis
Yeah. So, you’re champion in the one. And how did the other one go?

Annie Duke
I think I got to like the semifinals maybe.

Pete Mockaitis
Well, so that’s what I find so intriguing. It suggests that if it’s repeatable that you’re doing well, then it seems like there’s more than pure chance at work here.

Annie Duke
Well, I think it’s probably just, you know, I played a lot of poker so I sort of crawl into people’s heads a lot. And so, I think that I’m probably maybe better than the average Joe of figuring out what your patterns are, what you’re likely to be doing. And if you can do that, obviously, you can defend against it. But then you also have to have this kind of second-order knowledge of, “What if I’m against somebody who might be better than me at that?” then you can go to a random strategy.

And I think what happened was, I think I lost in the semifinals or the finals, but it was starting in the semifinals, or the round before that, that I used the random strategy. And I know I won one or two rounds with the random strategy where I felt like I’d come across somebody who was really good. And then, by the way, it really frustrates your opponent because they want to be able to apply their skill. And so, if they’re really good, then you take out a dollar bill, they realized that you’ve completely unarmed them.

Pete Mockaitis
Yeah. Well, that’s intriguing. And I read that they made a robot that can win rock paper scissors every time but it’s cheating. It’s like it catches what you’re going to do like a split second.

Annie Duke
Well, that’s not really winning now, is it?

Pete Mockaitis
Yeah, it could cheat at rock paper scissors perfectly.

Annie Duke
Great. Yeah, a cheating robot. You know what we really need to add to this dystopia right now? Cheating robots.

Pete Mockaitis
Cheating robots.

Annie Duke
We could just add cheating robots into the mix.

Pete Mockaitis
Oh, more headlines. More headlines to trigger anxiety. Okay. Cool. Well, that was fun. Let’s talk about decision-making when there’s more than…

Annie Duke
Well, we just did.

Pete Mockaitis
We did, how to win rock paper scissors under different circumstances. Well, so I love dorking out about decision-making tools. And I’d love it if, hey, there are some listeners who are not yet as enthused as you and I, can you make the case for the benefits professionals can enjoy with enhanced decision-making skills? And maybe, specifically, or particularly, for those who think that, “You know, I don’t have a lot of decision-making authority at my role. I kind of got to do what I’m told,” what are the benefits to be had by being excellent at decision-making?

Annie Duke
Let me give you just sort of the broader point, which is there’s only two things that determine how your life turns out, and it’s left in the quality of your decision-making. That’s it. So, there’s a whole bunch of luck that happens in your life, like, “What year are you born in?” It matters that I was not born in 1600 for the outcome of my life. And, obviously, from my perspective or from your perspective, coronavirus is a matter of luck. I assume you did not create the virus and distribute it.

Pete Mockaitis
Sure.

Annie Duke
But maybe that’s a bad assumption.

Pete Mockaitis
Certainly. And I guess there’s decisions you make associated with how much you’re going to go out, what measure you’re taking.

Annie Duke
Right. Exactly. So, that’s a good example, the two things that matter. There’s a whole bunch of luck that has to do with coronavirus, like the wrong human, to steal that line from Contagion. Then there are decisions that you can make given that that luck has occurred, and that’s the only thing that you have control over. And the better that your decisions are, the better your life is going to turn out.

So, I mean, that’s literally the simplest argument, which is it’s the one thing that you have control over that will actually have an actual real impact on the way that your life is going to turn out. Now, I understand that someone may, in a business setting, not be the ultimate decider, but the better your decisions, the more likely that you’re going to accomplish your goals within that environment. And there’s a few ways that you can think about it.

One is, of course, that you’re responsible for your own decisions. And one would hope that the better your decisions are, the more it maps onto your ability to actually move up the ladder or accomplish the goals that you’re trying to get to professionally. And you want to become more educated, and you want to implement a better process just literally for yourself. That’s number one.

Number two, there are certain things, there are certain behaviors that you can engage in that actually will start to get implemented in the people around you. In other words, you do have some influence even if you’re not the ultimate decider. You have some influence over the people around you that you can start to sort of get some of these really good decision-making skills and tools into a group setting.

And the last thing is, honestly, like, let’s say that I’m in a crappy situation with a bad boss, and they don’t really listen to anybody, and I don’t like the situation I’m in, that’s actually, in some ways, a more important time to be a good decision-maker because you need to be able to navigate those situations well. You need to decide when you want to stay or when you want to go, “Do I want to quit? Do I not want to quit? What can I do about this to make my situation better and actually to be able to thrive in an environment that’s an unhappy environment?”

Because, in a variety of ways, we can all end up in environments that are really unhappy where there are external forces that are making it very hard for us to thrive. And, while that is true and we want to be able to work to be able to change the situation that we’re in as much as we can, sometimes we have very little control over that, so you want to sort of grab onto like, “What are the things that I do actually have control over and improve those?” because those little changes will compound over time.

Pete Mockaitis
Well, that’s exciting. And so then, it sounds about as important as it could get in terms of what we can control that will impact everything in life, in career and happiness, decision-making enhances. So, could you maybe inspire us? Could you share a story of someone who, they thought their decision-making was fine, but then they adopted some of your tools and approaches and, boy, they saw some awesome results with their enhanced decision-making?

Annie Duke
If I were to think about this from prior to getting into a business setting, from a poker setting, the fact is that in order for me to improve my poker playing, what I have to do is to be able to think about, “What kind of were my predictions of the world?” and then try to figure out, “How did the actual outcomes that I got mapped onto my predictions of the world, what were the other ways that I might’ve thought about the hand?” And then I need to be able to talk to people in a way that’s going to expose to me the ways in which they may have differences of opinion with me, because the differences of opinion are where things get really interesting, right? Like, if you and I believe the earth is round, that’s pretty interesting, like, “Okay, the earth is round.” You’ll find that out.

Pete Mockaitis
“I also agree.” Conversation over.

Annie Duke
Yeah. But if I found that you think it’s flat, and I think it’s round, that’s like a humongous opportunity. And your listeners may be saying, like, “Well, how is that an opportunity for the person who believes that the earth is round?” which is a very common response for that. Isn’t that only an opportunity for the person who thinks the earth is flat? And I have a couple of answers to that.

Number one is things aren’t usually as clear as “We know that the earth is round, not flat.” We’re usually talking about things that are much more subjective, like which candidate to hire. And you believe we should hire candidate B, and I think I should hire candidate A, and we don’t know what the truth is, right? Not in the same way of round and flat, and so we need to have that discussion in order to get to the discovery that the earth is round. That’s the first piece.

But the second piece is that even when we hold opinions that are generally maybe are even true, it’s actually helpful for me to actually have to defend those against somebody that believes that the earth is flat. I don’t know about you but my arguments for why the earth is round would be super weak, like, things like, “Scientists say so, and I saw the pictures,” which are not particularly good arguments.

So, by having to actually be able to explain it to you, I’m actually going to know my own position better. So, what I was trying to do as a poker player was actually find out where there are areas of disagreement. So, when I actually work with teams, most of what I’m trying to do is that, and that’s how we’re improving decisions because what we’re doing is we have processes that are in place by which we can talk about, which allow for you to surface the dispersion of opinion as opposed to linger over the agreement.

Now, I’m sure you’ve been in lots of meetings where basically what happens is somebody says something and then everybody goes around the room and says, “I just want to double-click on what Pete said because I have my own reasons for believing the thing that he said, and I also would like to reiterate the same reasons that he said those things.” And you sort of go around the room, and then I guess everybody feels pretty good about themselves. But what you’ve really done is said, “I think the earth is round,” “I think the earth is round,” “I think the earth is round,” “I think the earth is round,” which is not particularly good for informing a group. It’s not good for informing a decision. It’s not going to actually improve decision-making at all.

So, what I’m trying to do with groups is get them to surface the areas where they disagree, where there’s actual dispersion of opinion, and then spend most of their time on that, really exploring that. By the way, not with the goal that they end up agreeing because when you’re talking about subjective things, like candidate A or candidate B, you actually shouldn’t expect agreement. And if you do get to agreement, probably somebody is actually not agreeing, they’re giving in, which is a really different thing. But we want those different viewpoints to collide, and then that really improves the decision-making.

Now, it turns out that when you really do a good job of surfacing the dispersion in the first place, you also create this amazing record of why you think what you do, why you want the decision that you want, what you think is going to be true of the world in the future. And this, then, has a huge impact on your decision-making because, after the world starts to unfold, as it does, like after the future starts to happen and become the present, you’ll have like an evidentiary record that you can go back and look at. And this now allows us to actually create really nice closed feedback loops where we actually know what we’re supposed to be looking for in order to become better calibrated in our decisions.

So, what I can tell you is that the groups that I work with, when we actually get these kinds of processes implemented, the quality of the conversation shoots through the roof, meetings are shorter, but more informative, which I think everybody would really like. And then the way that they’re actually thinking about dispersion, like, “What does it mean for somebody to disagree with you?” moves out of sort of the defensive world into the open-minded world because it really reinforces these ideas that the goal of a meeting is to inform not to agree. And then it actually helps them to much more quickly to recalibrate if their calibration is off because you can close these feedback loops really quickly, and actually more accurately.

Pete Mockaitis
Well, I love that point you brought up about defensiveness there, and even the phrase dispersion of opinion, you know, feels emotionally a lot more comfortable than disagreement or conflicts.

Annie Duke
Well, that’s why I’m using that term actually.

Pete Mockaitis
Masterful. Good work.

Annie Duke
Yeah. So, it’s in my book, and I really recommend that people start to use this term, dispersion or divergence. Both of those words, I think, are really good. Where do we diverge? And where do we converge? Because I think disagreement has such a negative connotation. It sounds so combative. And when I feel like you disagree with me, it gets translated for us sort of just cognitively into like you’re attacking my identity as opposed to just like, “Oh, we have a disagreement about these things.” It feels like an attack on my identity.

And, generally, what happens is that when I view it through the lens of disagreement, I’m going to tend to shift into convince mode as opposed to convey mode. In other words, I’m going to want to bring you over to my side of the argument in order to certify my beliefs and certify my identity, and so the way that I’m speaking to you is going to be meant to convince. It’s going to create a lot of interrupting, me saying, “Well, have you thought about this? So, you weren’t thinking about this data, or I think you’re wrong about this,” and so on and so forth. As opposed to like a real honest exploration of me trying to understand why you believe what you do.

Pete Mockaitis
Yeah. Boy, it’s funny, this reminds me of a time when it was way back, I think maybe in high school, in which I was arguing with somebody, and we had some friends and we just decided that they were going to be the jury, and we would make our case and advocate for our perspective. And it was kind of funny, it was kind of a joke, but it got a bit heated actually. And then when the jury left, it was just the two of us, and we just sort of chatted out with a completely different intention of, “Well, let’s sort of really see what kind of went on there and what we should do about it.” And it was just sort of like night and day in terms of “Are we trying to convince to win the argument as opposed to kind of collaboratively jointly discover what’s as accurate as possible?”

Annie Duke
Right. Yeah, exactly. And I think that the other thing that we need to realize when we’re dealing with things that are in the subjective world, so we’re not talking about “2+2=4” or, “The earth is round.” For most of the decisions we’re making in our lives and in a business setting, by the way, we’re talking about things where we’re trying to discover what is subjectively true, but what is subjectively true is not known so we’re having to go through the discovery process in order to get there.

And so, the idea that you somehow know the truth and you need to convince other people of your side is really, really unproductive, and it’s going to create that kind of thing. It actually makes more sense that the two of you convey why you believe what you do, and then you can walk away not agreeing. And that’s okay because you don’t need to.

If you think about, for example, if you and I are in a hiring committee, and I really care about whether I think the person is going to be a generous team member, like cooperative, generous, someone who doesn’t take credit for themselves but likes to share credit and things like that, and you care, all you care about is what their sales production is, right? Literally, you’re just a numbers person, right? That’s okay.

I don’t need to convince you of what my values are and you don’t need to convince me of what your values are because, by allowing those two perspectives to just sort of live and breathe, and for me to express why I believe what I do and why I think that’s important, and you can express what you believe and why you think that’s important, we’re probably going to hire a better candidate, because what’s going to happen is that’s now going to get expressed in our hiring rubric and who we actually end up bringing in.

Pete Mockaitis
Yeah, I like that a lot. Well, so we’ve already covered some great tools and perspectives associated with in-group settings, how we can view it as a dispersion of opinion or divergence as oppose to a disagreement, and how we’re not trying to convince but to convey, and we’re all enriched as a result of having engaged in that.

I’d love to zoom into if it’s sort of an individual and it’s sort of I’ve got one person making decisions for himself or herself, and doing the research, and there’s not so much a collaborative exercise going on, what are some of the best tools in this context to make better individual decisions?

Annie Duke
Well, first of all, not a pros and cons list.

Pete Mockaitis
Okay.

Annie Duke
Which I think most people might find kind of surprising, I know that. So, the thing about a really good decision tool, like if we were to think about what’s a great tool, decision tool or otherwise, like, if we think about a screwdriver, right, it should be accomplishing the purpose that it’s meant to accomplish. So, like if I want to get a screw to actually go in the wall in a way that’s going to be safe and actually accomplish the job…

Pete Mockaitis
Ergonomic. Convenient.

Annie Duke
Yes. Which is why I want to be able to use a screwdriver as opposed to a hammer or a jack hammer. So I want the right tool for the right purpose. But here’s also the really important thing about a tool is that I need to be able to repeat the use in a way that’s going to create really high fidelity. And then I also need to be able to hand it to somebody else and then explain it to them so that they could actually use that tool in the exact same way.

So, when we sort of understand that we see where decision tools really go awry. So, like, “Your gut is not a decision tool.” “Well, why?” “Because I can’t actually look at it and explain it to you, right?” That’s where we’ll go. “Well, my gut told me so,” and you’re like, “Okay, but that doesn’t really…I can’t use your gut.” Right? But you know what I mean. It’s like, “Okay, but I can’t actually examine to see whether you screwed that in well, and then you can’t explain to me exactly how you got that screw in the wall, or what you were doing. And I can’t actually repeat that process because it’s a black box.”

So, a pros and cons list, in some sense, certainly is a tool in the sense that we know its purpose is to get you to decide about whether you want to proceed with an option. And I could actually sort of teach you it in a structural sense. So, that’s all okay. So, we’re getting part of the way there. It’s certainly better than gut. But here’s what that tool lacks that will actually reveal what the kinds of tools are that we actually want to be using.

So, the first thing that it lacks is that it’s a list, literally a list, which means that it’s flat. So, what do I mean by flat? It’s flat in two ways. One is that when we think about something that’s on the pro side or something on the con side, we don’t have a sense of the magnitude. So, it could be like I could get a hangnail and I could die. So, those are both there, because all I sort of have is this list.

And so, that’s one of the first problems is that sort of the magnitude of how positive the things on the pro side are, in terms of achieving your goals, is not actually anywhere explicit in the list, and the magnitude of how negative the cons are, it’s also not existing in the list. So, that gives us hint number one, is that we want to have an idea of this magnitude if we’re going to have a really good decision tool.

The second piece is that we also don’t have a sense of the probability of those things occurring.

Pete Mockaitis
Yeah.

Annie Duke
So, if we have a con, that’s like, “Well, I could lose $10,000,” you would want to know, “But how likely is that to occur?” Right? So, you could have a pro, which is like, “I could win a million dollars.” You could do this with the lottery, right? But the con could be, “I’m going to lose a dollar or two dollars,” and the pro is, “I could win the jackpot, so maybe that looks pretty good.” But what we need to understand is, “What’s the probability of winning the jackpot?” which is de minimis, versus “What’s the probability of me losing the two dollars?” which is basically every time.

And if we don’t have that information, it’s also incredibly hard to compare. So, when we see that, what happens is it becomes very hard to understand whether an option is good or not, and then we get into the problem of how on earth would you compare options. Like, if I had one option that had 10 cons and 2 pros, would that be worse than an option that had 5 cons and 4 pros? Well, I don’t know because I don’t know what the magnitude of those pros are and cons, and I don’t know the probability of those things occurring is, so it’s hard for me to compare.

And then we have this added issue, which is that it’s basically, literally, a tool for expressing your bias, like your cognitive bias, because you can imagine that you can take something that could sort of be one pro or one con, and you could divide it up into its little bits in order to create ten ways to express that. So, the con could be like, “Well, I might end up like really unhappy,” so that would be one, but it could also be like…

Pete Mockaitis
“I could be anxious. I could be stressed. I could be disappointed.”

Annie Duke
Exactly. Right. And now, all of a sudden, it’s ten things, right?

Pete Mockaitis
Yeah.

Annie Duke
So, what ends up happening is that as we’re sort of exploring those pros and cons, generally, as we’re entering into a decision, we’re already sort of somewhere in our head kind of know what our opinion is and know what we would like to be true, and then we do the pro and con list, and all it’s doing is kind of like expressing whatever that opinion already is, but it’s certifying it as objective when it’s not actually objectively. And that’s actually a super bad combination.

And you can see how this is a problem, like particularly if we’re trying to compare options because we’re going to do it just by list. And so, the option we don’t want to do, we can just create a lot of cons for. The option that we do want to do, we want to create a lot of pros for. So, that’s sort of through the negative frame of like, “Here’s a tool that everybody really understands,” that turns out to be sort of the equivalent of taking a jack hammer to get a screw in the wall. Okay, so we don’t want to do that. We’re going to ruin the wall.

So, that tells us, “Okay, so what does a good decision process going to do?” Well, it’s going to solve this problem of sort of dimensionality. So, for any option we’re considering, we want to think about what the likely outcomes of that option are. But then we want to think about how much is that option going to advance us toward our goal or way. So, that gets that idea of the payoff, what’s the magnitude of how good or bad we consider that option is for us. But then we want to take a stab at what the likelihood of those things occurring is.

And what that allows us to do is understand, for example, like in the startup world, you may have a really high likelihood of failure but the payoff is so large that if that payoff is likely enough, you would still do it despite the fact that mostly it’s going to be bad outcomes. But that’s okay because we’ve added this likelihood piece in, and we’ve added sort of like what does the payoff look, and we can start to bring that into our decision-making. And you can see that that now gives us a real way to compare our two options, because now we have a pretty clear sense of what’s the upside potential and the downside potential, and, “Does the upside outweigh the downside given whatever I’m willing to risk?” And then I can now compare those two things.

So, like a simple example would be, like let’s say that I have two candidates that I’m thinking about hiring, A and B, and I really, really care about retention, like my recruitment costs are out of control and I’ve got all these employee turnover, so this is something that I happen to be focusing on. And so, what I can do is I can say, “I want to think about kind of these three buckets that the person that I’m hiring is going to be with the company between zero to six months, six months to 18 months, beyond 18 months. Let’s say that we set those three things up.

And then, basically, what I can do is just have anybody on the hiring committee, for any candidate that we see, to say, “What do you think the probability of those three buckets is?” because that’s what I really care about, right? And now I actually have an apples-to-apples comparison. So, I’ve thought about, “What are my values? What are the payoffs that I’m trying to get? I want this person to stay here a long time. And I’m looking for the person who is going to stay here the longest. That’s what I care about.” And now I have a way to actually compare options.

Pete Mockaitis
Okay, so we covered some of the shortcomings of the pro-con list, we got it captured very clearly the magnitude of things, and the probability of those things occurring. And so then, I’m imagining kind of just a spreadsheet by this point in terms of I’ve got a few things, I’ve got some magnitude, I’ve got some probabilities. And I guess it gets a little tricky if it’s not just money in terms of like, “How do I put a number on my stress?” How do you do that?

Annie Duke
Well, so I think that it’s really interesting. When we get into things that we feel are more subjective, we think that we actually don’t know anything and so, therefore, we shouldn’t try, “What’s the probability I’m going to be stressed?” Or it doesn’t even have to be something that’s like so clearly subjective like stress, but like what’s the probability a candidate is going to be with a company, is going to leave within six months? Well, we don’t know. We’ve never hired that candidate before.

So, in the sense of, “Can I be exact?” or if I’m releasing a software feature and I want to know, like, “Oh, of the people who use my product, how many of them are going to start adopting this, like, the daily users of this new feature within the first month?” Obviously, these aren’t things that are like 2+2=4, and they’re not things like if I flipped a coin, it’s going to land heads 50% of the time where like I know for sure what the answer is because we have enough information.

What people end up doing in that case is very often just saying, “Well, I’m not going to try because I can’t come up with ‘the right answer.’”

And the problem with that is that then we just sort of get we get mired in the limitations of our own sort of lack of knowledge instead of thinking about, “Well, I want to be an educated guesser, and my goal as a decision-maker is actually to get more educated because I have all these uncertainty in trying to forecast the future?” which is really what we’re doing when we’re saying, “What are the possibilities or the probabilities and things like that?”

There’s all this uncertainty in my ability to forecast the future, but the more educated I am, while I may never get perfect, I’m going to get closer to the range of what is objectively true if I were omniscient, and that’s actually going to improve my decision-making. So, I can do an example of this with you. My computer is sitting on a stack of books. Now, obviously, you can’t see the books because it’s what my computer is sitting on. I’m on the computer looking at each other, so you don’t know how high the books are and you don’t know what type and you don’t know what number, right?

Pete Mockaitis
Right.

Annie Duke
Okay. So, how much does the stack of books weigh?

Pete Mockaitis
About five pounds.

Annie Duke
Okay. And what do you think the lowest amount of the stack of books weighs is? Do you think it’s possible this stack of books that it’s sitting on could weigh a pound?

Pete Mockaitis
I think it’d be improbable that a stack, implying multiple books, weighs less than one pound.

Annie Duke
Okay. Could it weigh 200 pounds?

Pete Mockaitis
No.

Annie Duke
Okay. So, I think this is really good, right? So, what we discovered is that you could’ve said, “I don’t know.” But what I just did was I said, “Well, but you know things about books.” And so, while you may not get the exact answer, you’re going to get an answer that eliminates a huge number of possibilities. In other words, it’s going to get you somewhere closer to what’s actually true of the stack of books that my computer is sitting on. And that’s a really important exercise and it’s a really important exercise for three reasons that I hinted at.

Reason number one is that the more accurately you’re thinking about the future, in other words, “Can you get in a target range?” Like, if you think about it like an archer. And, in fact, in the book I talk about like the archer’s mindset, right? Yes, you’d like to hit the bullseye but you get points for hitting the target. And the closer that you can get to hitting that bullseye the better off you are, but you’re still getting points. It’s like you still get points for showing your work, right?

So, even if you hit the outer edge of the target, you still get points because all the stuff that isn’t on the target, like you know that these books don’t weigh 200 pounds, is going to help you to actually have better decision quality because you’re eliminating all these different possibilities that the answer could be that’s going to clarify your decision and get you better at sort of calculating, really, in the end what’s the expected value of the decision. Like, how much upside potential compared to downside potential do I really think there is? So, that’s number one is that you’re going to be creating a more accurate view of the future even if it’s not perfect, and that’s good.

The second thing is that, which I had hinted at before, is that we have this problem as decision-makers, which is, generally, the stuff that we know is like so tiny it could fit on the head of a pin compared to the stuff we don’t know, which is like the size of the universe. Obviously, if you have the ideal decision tool, which I think would be a crystal ball, you would be set because that universe stuff that you didn’t know would be revealed to you in this psychic instrument that you have that caused an omniscience and an ability to foresee the future, but we don’t have a crystal ball. So, what we’re really trying to do is, “How can we create a set of tools that will allow us to cobble together something that is crystal ball-like?” And part of that is dealing with this problem that there’s this whole universe of stuff that I don’t know.

And by forcing yourself to guess, I made you think about that. I made you think, “What do I know about books?” so you’re exploring that world of things that you do know in order to try to make yourself get the educated into the guess, and then you may, in other cases, start thinking, “Well, what is the universe of stuff that I don’t know? And maybe that would actually help me with my guess.” So, like if we went back to something as simple as a hiring example. One of the things that we might do is say, “Well, maybe I could go find out how many candidates, like when companies hire into this particular position, what the average retention in the industry is.” That’s called a base rate. And that would be incredibly helpful for me to go find out as I’m trying to estimate what I think any candidate that I might see is.

Now, that doesn’t mean that the candidates I see are going to be right there on the base rate, but it’s going to give me a place to anchor to about kind of what’s true of the world in general that’s really going to help me. The other thing that I might do is to go ask for somebody else’s perspective where we know that two people can be looking at the exact same data and come to very different conclusions about it, right? So, I could ask one person, “What do you think these books weigh?” and then I could ask somebody else, “What do you think these books weigh?” And maybe you said five pounds, maybe they say 20 pounds. Great. Now, we go back to that earth is round and flat thing, and now I get Pete who’s the five-pound person and Susan who’s the 20-pound person to have a discussion about why they have that dispersion of opinion that’s probably going to get me closer to what the most educated answer would be, closer to what’s objectively true of the world. And that actually like incredibly important.

So, whether you’re forecasting, like, “What’s my stress level going to be?” or, “How long is someone going to be with the company?” or, “How many users are going to adopt this on a daily basis within the first month?” all of these things, which we’re lacking information about, not allowing yourself, “Well, how could I ever know that?” and not accepting that as an answer, is actually really crucial to a good decision process.

Pete Mockaitis
Certainly. And I think that’s a great perspective in terms of you don’t know it exactly but knowing it’s more than one pound and less than 20 pounds is way, way more narrow than it could be anything.

Annie Duke
Oh, my gosh. Right. And I think I make the point in the book that this is part of the reason why we want to communicate with precision.

So, I think I make the point that if I say 2+2 is a small number, I’m technically correct but it’s going to be harder for you to tell me things that might help correct my inaccuracy is because the target area is kind of broad that I’ve given you, and it’s going to be hard for me to get better at math. Now, I’m going to get somewhat better because if I say 2+2 is a very large number, you’re going to be able to correct that. So, it’s not that I can’t improve, but it’s going to slow down my improvement that I’m not willing to give an exact answer, like 4, right? And there’s ways, obviously, if I’m not being precise that I can game it because I can say 2+2 is somewhere between minus infinity and positive infinity and, okay, I’m technically right. But what is that value of the information there in terms of actually improving my decision-making because, if you think about it, this is the reason why a crystal ball would be such an amazing decision tool is because all decision-making is forecasting of the future.

When I make a choice, when I pick an option, what I’m saying is that, “I think that given whatever goals I have and what my values are and my resources are that this option is going to be the most likely to create the type of future that I would like to unfold, and so I am being like a soothsayer in that sense. I’m making a prediction about the future.” And what we’re trying to do is make those predictions higher quality.

Pete Mockaitis
That’s great. Well, tell me, Annie, anything else you want to make sure to mention before we shift gears and hear about some of your favorite things?

Annie Duke
Yeah, I think I’d like to just say, like, just one really important decision tool, when we’re thinking about, “How are we actually getting a better view of the future? How do we actually become better fortune tellers?” Those are what we’re trying to do. And I just want to give a real pitch for a decision tool that I think is somewhat counterintuitive, at least in popular culture, which is the power of negative thinking as opposed to the power of positive thinking.

So, the power of positive thinking is like so incredibly powerful in the literature from Napoleon Hill.

We know about The Power of Positive Thinking, and it’s very popular which is you imagine a destination that you’d like to get to, and then you imagine success along the way. And I think that that’s a really bad decision tool, and I’m not saying that people should not imagine positive goals. Of course, you should. But the whole key to unlocking decision-making is to imagine the obstacles, the ways in which you might fail along the way. Why? Because that is the only way you can avoid them.

So, the way that I kind of think about it is the difference between a paper map and Waze. A paper map, you look at the destination you want to get to and then its clear roads. And I think about that as the power of positive thinking, right? Like, “Here are the clear roads, and now I’m just going to go along my merry way along those roads.” But what does Waze do? Waze says, “Here’s the destination you want to get to. And, by the way, there’s a road closure over here, and there’s like an accident on this one, and there’s heavy traffic over here, and so I’m going to reroute you so that you can actually successfully get to your destination.”

And I think the problem with the positive thinking literature is that sometimes it’s explicitly stated when you get into some sort of cookier versions of it, like The Secret, but it’s certainly implied in all of it that if you imagine failure, that it’ll actually create failure. But what an app like Waze tells us is that if you imagine failure, it actually creates success because that is the only way that you can get out ahead of it. And the more that you can identify the obstacles that might lie in your path, the better off you’re going to be because you’re going to have a clear view of the future, and you’re going to have a clear view of the kinds of things that you might want to avoid, the kind of things that might get in your way.

So, one of the best decision tools that you can use is called a premortem. And it was originally developed by Gary Kline. I have an adapted version of it in the book. And, essentially, what it asks you to do is to imagine a goal or a decision that you’re making which has an implied goal that it will work out, and imagine that it’s however long it would take for you to know whether you’ve reached a goal. So, let’s say that you have a goal to increase sales by 10% in the next year. And so, you imagine it, a year and a date from now, and you failed to reach that goal, and you ask yourself, “Okay, why did that happen? Why did I fail?” And you divide it into two categories: matters of your own decision-making, “What are the decisions that I made that may have led to this failure?” and then matters of luck.

And, as I recommend with everything, you try to figure out how likely those things are, and then you can actually figure out what to do about it. You may say, “Maybe I should change my goal,” or you may keep your goal, and you say, “Well, here are a bunch of decisions that I might make that really would cause me to fail, so let me try to figure out how not to make those so that I don’t actually engage in these kinds of behaviors.” If I want to lose weight, I have to figure out a way, because I know a point of failure is people bringing in cupcakes for their birthday. I need to figure out a way to not eat the cupcakes when that happens. I need to see that that’s on the horizon, and actually try to figure out how to avoid it.

And then with matters of luck, you can think about, “Are there ways, are there decisions that I can make that can reduce the probability of these bad things happening?” I can’t control the luck but I might be able to reduce the probability of those things occurring. And even if I can’t, maybe I can have a plan for it so that I’m not just running around like a chicken with my head cut off and so I can figure out what those are. And maybe I can find a hedge which is just like buying stocks and bonds at the same time. And if you don’t actually think about, “How can I instantiate this idea of sort plan positive, think negative?” into your decision process, you’re going to be constantly surprised by the world. You’re going to be using a paper map when everybody else is using really solid GPS. And we know that people who use paper maps have a disadvantage in terms of getting to destinations on time than people who use Waze, so don’t be the person still using a paper map as it applies to your own decision-making.

Pete Mockaitis
Beautiful. Thank you. Well, now, could you share a quote, something you find inspiring?

Annie Duke
My favorite quote from Feynman just has to do with him saying, “If you can’t explain it to a child, you don’t actually know it yourself.” And this is a paraphrase of the quote obviously. But the reason why I like that so much is that it kind of really has to do with this idea of what makes for a really good tool, is I have to be able to explain it to you, and I have to explain it in simple terms.

And what I really love about that sort of second piece of not just, “Do I need to be able to hand you the screwdriver so that you can use it, but if I can’t explain it to you, I don’t really understand how to use a screwdriver.” And if I can’t do that, I butt up against the limits of what I know in a way that when we talk about that universal stuff we don’t know that we really want to be exploring, it makes me go look in that universe, and then I think it expands my knowledge, and everybody is better off for it because I explained to you how to use a screwdriver, and then I understand screwdrivers much better for having had to go through that process. And that’s why I love that Feynman quote so much.

Pete Mockaitis
And you might think I already know to screw nails on, or screw a screw, but sure enough you say, “You may have better experiences in terms of stripping them less often, giving them straight the first time, not having to redo stuff.”

Annie Duke
Right. When people are having success doing something, and they don’t start thinking about “What are the limits of my knowledge? And what are the limitations of the way that I’m thinking about this and my perspectives on the world?” what happens is that they get disrupted from without, and you’d rather be disrupted from within. So, you can look at IBM in the 1980s versus a Microsoft or Apple, and this is a big danger when you’re doing things pretty well, and your models of the world are pretty good.

But just as we talked about with things that are subjective, your model can be pretty good and it can be working, but that doesn’t mean that you have the objective truth. Like, you want to be exploring different ways that people could be looking at the problem, and always seeking new knowledge, and always sort of testing your ideas to see if there isn’t a better way, and also, sort of back to the idea of negative thinking and that causes you to have to sort of explore the limits of your own knowledge and your own ideas in a way that’s actually going to help you to improve them and disrupt your own ideas instead of allowing someone else to come in and disrupt you, which is something that we’re all trying to avoid.

Pete Mockaitis
And how about a favorite book?

Annie Duke
I’m just going to answer it by recent, right? So, I’m going to give you two favorite books right now, and then two that you should be looking out on the horizon. My two favorite books right now are Maria Konnikova, which is The Biggest Bluff which is amazing. It’s like a marriage of memoir and exploration of the influence of luck in your life. So, Maria decided she wanted to explore luck because she had just sort of stuff happened to her. Like, her husband lost his job, she got sick, I think one of her grandparents died, sort of like all at once, and she’s like, “Whoa,” and she wanted to explore it. So, she said she’s going to learn how to play poker from being a total novice.

She ended up really doing well. She won a huge poker tournament, and, it’s this really wonderful book. It’s really beautifully written and it’s a great exploration of just sort of the influence of luck in your life.

The other book that I’m really recommending right now is The Psychology of Money, which is by Morgan Housel, he’s so good with just kind of like taking really complex concepts and making them very understandable through really, really fun narrative. And he’s really just talking about, like, “What are the different ways that we think about money?” Like, what is money? It’s sort of an object that we can sort of explore and understand, like, “What is its purpose in our life? And how do we think about it? And what should we do about it and do with it?” It’s just a really fun book. I really think that everybody should be reading that book.

In terms of books on the horizon to have, to be on the lookout for. Katie Milkman, who’s a professor at Wharton, and has a book coming out in the spring called How to Change, which is incredible on just if you want to create better habits in your life, just understanding, “When does habit change occur? Why? What are the ways that you can sort of make that happen for yourself?” It’s a really wonderful book. It’s really fun.

And then Noise is going to be coming out soon from Kahneman, Cass Sunstein, Oliver Sibony, and I’m really excited about that. It’s like a contrast to Thinking, Fast and Slow which is more about cognitive bias, and this is just more about sort of noisiness in the system, and it’s a really good book. So, those are two for the horizon. And even winnowing it down, I gave you four, so I’m…

Pete Mockaitis
That’s great. Thank you. And do you have a final challenge or call to action for folks looking to be awesome at their jobs?

Annie Duke
I would like people to practice, when soliciting opinions or feedback from somebody else, to try to not offer their own opinion first and see what happens. So, there’s this really big problem, like when we’re talking in the meeting sense about we all think that the goal of a meeting is to agree. That’s true one-on-one as well. It feels really to agree with people that you’re talking to, that’s why we end up in echo chambers.

So, your opinions are contagious. So, if I want to know what you think about like Perry Mason, which is on HBO, if I really want to know what you think, I should just say, “What do you think about Perry Mason?” But what we do is we say, “Oh, I watch Perry Mason. I thought it was really cool and interesting, and I think it was really fun to see his journey from detective to lawyer, and I like it that he was a flawed character as opposed to the Raymond Burr version. What do you think?” And that’s obviously something simple about a TV show that probably isn’t very impactful. But think about that in terms of when you’re really trying to get somebody’s help, is I’m not actually going to get your true perspective.

When we talked about surfacing the dispersion of opinion, how am I going to surface the dispersion of opinion if I offer you mine first? So, I really challenge people to start trying to implement that into their own life, and I think they’ll find that it really changes the communication, and how much you sort of get to what people really believe that can really spur these interesting conversations.

Pete Mockaitis
Excellent. Thank you. Well, Annie, I wish you lots of luck with your book How to Decide and all your decision adventures.

Annie Duke
Well, thank you very much. I’m so happy that we got to talk again.