Rob Alvarez
, , , , , ,
0

Personalization Is Overrated, with Kristen Berman | Episode 463

  Listen on SpotifyListen on Apple PodcastsAvailable on YouTube Don’t experiment on your own revenue with broken game mechanics. Get our guide “Core Drives in the Wild” to learn how to apply real behavioral science to your product: professorgame.com/WildCD

Episode Summary

Kristen Berman, behavioral scientist and CEO and co-founder of Irrational Labs, argues that personalization is overrated in product design, and she brings the experiment to back it up. She walks through a published onboarding test with Headspace where asking new users a few questions and then recommending the same Basics course to everyone doubled course enrollments, outperforming both the personalized recommendation and the version with no questions at all. She is just as direct about the limit: enrollments doubled, yet people did not meditate more, because the rest of the product never reinforced that first commitment. Listeners come away with a barriers-and-benefits lens for any signup flow and a clear order of operations: get the fundamentals of human decision making right first, then layer personalization on top.

About the Host

Rob Alvarez is Head of Engagement Strategy, Europe at The Octalysis Group (TOG), a leading gamification and behavioral design consultancy. A globally recognized gamification strategist and TEDx speaker, he founded and hosts Professor Game, the #1 gamification podcast, and has interviewed hundreds of global experts. He designs evidence-based engagement systems that drive motivation, loyalty, and results, and teaches LEGO® SERIOUS PLAY® and gamification at top institutions including IE Business School, EFMD, and EBS University across Europe, the Americas, and Asia.

Key Takeaways

  • Headspace’s onboarding experiment with Irrational Labs doubled course enrollments by asking new users a few questions and then recommending the same Basics course to everyone, with no personalization behind the answers.
  • Personalization pays off when the match is right and loses the user when it misses, and a handful of signup questions rarely reveals enough to get it right. Kristen Berman’s advice: design first to avoid getting it wrong.
  • Doubling Headspace course enrollments did not get more people to meditate. The enrolled course sat at the bottom of a list instead of on the Today tab, which shows that fixing one step of the funnel does not fix the next one.
  • Forced choice beats a Continue button: in an Irrational Labs experiment, showing Accept or Decline led more people to accept, because the cost of saying no becomes visible instead of staying hidden behind an X.
  • A meta-analysis from around 2020 found no effect of personalized health feedback on behaviors ranging from smoking to physical activity. With reminders, whether you send them and how often matters far more than small copy changes.
  • Duolingo times reminders to the hour a user signed up, and Netflix improved its recommendations by moving from five stars to thumbs up or down so more people rated. Past behavior is the most reliable personalization signal.

Topics Covered

  • 0:00 – Doubled enrollments, no more meditation
  • 0:13 – Meet Kristen Berman of Irrational Labs
  • 2:56 – Where irrational decisions cost the most
  • 5:06 – Why game designers get to be paternalistic
  • 7:48 – Forced choice: accept or decline beats continue
  • 10:02 – Friction versus motivation in fintech and health
  • 13:58 – The Headspace onboarding experiment setup
  • 17:45 – Non-personalized questions doubled course enrollments
  • 19:46 – Why more enrollments did not mean more meditation
  • 22:21 – Pre-commitment, lost streaks and why willpower fails
  • 25:24 – AI personalization and the health meta-analysis
  • 28:44 – Reminders, Duolingo timing and Netflix ratings
  • 32:08 – Where to find Kristen Berman
Don’t experiment on your own revenue with broken game mechanics. Get our guide “Core Drives in the Wild” to learn how to apply real behavioral science to your product: professorgame.com/WildCD

About Kristen Berman

Kristen Berman is a prominent behavioral scientist and the CEO and co-founder of Irrational Labs, a leading behavioral design firm in Silicon Valley. Irrational Labs partners with top companies such as Intuit, LinkedIn, Airbnb, Meta, and Microsoft to drive behavior change for the greater good. Kristen’s expertise in behavioral economics began as a founding member of the behavioral economics group at Google, where she worked with over 25 teams to launch innovative interventions, and she went on to co-found Duke University‘s financial decision making lab, Common Cents. She also publishes Product Teardowns, a Substack where she takes apart a popular app in about five minutes to show how a behavioral scientist thinks about design, pricing, incentives, and loyalty.

Find the Guest Online

Mentioned in This Episode

Free Resources and Get in Touch

Looking forward to reading or hearing from you, Rob Full episode transcription (AI Generated)

Doubled enrollments, no more meditation

Kristen Berman (00:00): The one thing I would say is we got doubled course enrollments from this intervention, very impressive, and we did not get more people to meditate.

Meet Kristen Berman of Irrational Labs

Rob Alvarez (00:13): So Engagers, welcome back to another episode of the Professor Game Podcast. As you know, we are the number one gamification podcast, and we explore how not only games and gamification help us boost engagement, retention, and loyalty, but we also start have been starting more and more to look into the behavioral aspects with behavioral science, behavioral design, and today, very especially behavioral economics as well. We have a very, very special guest lined up with us today. And as you know, I’m Rob. I’m the founder and coach here at Professor Game. I also head engagement strategy at the Octalysis Group in Europe. As you know, we are a leading consultancy in behavioral design and gamification. Rob Alvarez (00:47): And I also teach gamification, behavioral design in IE Business School, EFMD, and other business schools around the world. And remember, if you want our quick Core Drive, Core Drives in the Wild guide for free, just click on the link below and we’ll give you a bunch of examples, ideas, and my own take on those subjects. Without further ado, Kristen. We need to know. Are you prepared to engage? Kristen Berman (01:08): Yeah. Rob Alvarez (01:09): Thank you. Let’s do this. We have Kristen Berman, a definitely a prominent behavioral scientist and a co-founder of Irrational Labs, a leading, and I said that right, Irrational Labs, yes, a leading behavioral design firm in Silicon Valley. They partner with top companies, including Intuit, LinkedIn, Airbnb, Meta, and Microsoft to drive behavior change for the greater good. Kristen’s expertise in behavioral economics has begun as a founding member of the behavioral economics group at Google, where she worked with over twenty-five teams to launch innovative interventions. And as a co-founder of Duke University’s financial decision making lab, Common Cents. Kristen, is there anything we we should be mentioning before we dive straight into the interview? Kristen Berman (01:50): You you nailed it. Let’s let’s do it. Rob Alvarez (01:53): So as you know, well actually let’s take a step back because I mentioned Irrational Labs and for some people who are not familiar with the lab maybe, can you give us a quick gist of what it is you guys do in the space? Kristen Berman (02:04): Sure. So our team of behavioral scientists, we study decision making, why people do what they do and many times why they don’t. And a theory of behavioral design, behavioral economics is that the environment may influence us more than we give it credit for. So in one world, we could say I’m gonna persuade you with good arguments and information to do something. In the behavioral economics, behavioral design world, we actually think it’s more impactful typically to change the environment of decision making. And this means specifically with digital environments to change how a user or consumer experiences a product or flow. And so our team looks in the very small details of how something is designed and we try to understand the psychology that goes into something and then help the user get to the thing that they likely already want to do in the product.

Where irrational decisions cost the most

Rob Alvarez (02:56): Amazing, amazing. Sounds very, very aligned as well, which I love. And Kristen, within this this whole work that you’ve been doing, and and I I like the name of of Irrational Labs because as as you clearly very well understand, we don’t really behave rationally, right? And a lot of people know this, a lot of product people know this, but still design as if we actually do behave very rationally, very logically. Have you seen that there’s a a a a set of places or something you could point to where that really, really hurts and you know, maybe what’s what’s the cost of that of that kind of hurt when people forget about this this stuff. Kristen Berman (03:35): Yeah. So it typically every day we’re making lots of decisions. And so if you go to the grocery store and you’re using some heuristic to make a decision, look, that’s fine. It may not be that you should be weighing the exact pros and cons, the amount of time and benefit that you would get from buying a tomato versus an apple, or going to the coffee store and thinking about the three dollars that you are spending on the coffee versus the three dollars you could invest in the market. These are small decisions that we are typically using heuristics for. and it’s fine. We we’ve learned many shortcuts here. Kristen Berman (04:11): We think that the main places that kind of this irrationality gets to us, or could be, you know, that that that decreases maybe the likelihood that you’re getting to the thing you actually want is in bigger decisions. So if you think about something like financial decision making where you put your money, or you think about something like health, are you going to the doctor or not? Or just time spent, how you’re behaving. online. And potentially when you sign up for a product or service, if you have the intent to do something and you’re just actually not doing it because your lives are really busy, your kids are yelling at you, you’re checking Instagram. Kristen Berman (04:44): these are things where you have some intent to do a thing and the rest of the world is getting in your way. And so really the opportunity for behavioral science or behavioral design is in those moments where the either a big decision or you just clearly have some intent for something and potentially you could have have more support there. I’ll I’ll also say one other thing is I

Why game designers get to be paternalistic

Kristen Berman (05:06): I think in general, you know, ga gaming actually tends to be fairly paternalistic. You’re not gonna let the user or the gamer do whatever they want in your game. You’re actually pushing them through some flow. And they may have a couple options of what to do, but you’ve designed those options for them. and I think that actually very much represents kind of the behavioral science view of the world is that Look, if you have all the options in the world, it’s likely you may choose the wrong thing because you’re not an expert at most things. We’re an expert at very few things. And so as a game designer, you get to kind of be paternalistic. Kristen Berman (05:38): And I think that is actually very, very similar to the behavioral science mindset. And we push a lot of our kind of product teams that we work with to be more paternalistic. They know more than the user about the opportunities in front of the user. And so if they can kind of direct the user a little bit more, this is usually successful. Rob Alvarez (05:54): And that’s that’s like when you arrive to to a restaurant, right? And I don’t know if everywhere it’s it’s the same way, but you arrive especially to here it’s Indian and Chinese restaurants, especially where you have like the menus like this thick. And it’s like, Well, I really don’t know. So there there is some heuristics. I’ve for me it kind of works most of the time. You go to last page and they have like the combinated com combined stuff and the that tends to be, you know, easier. I’m not gonna wade through every single page and I don’t really mo know most of the plates anyway. So I just go to the last page and that kind of works. Rob Alvarez (06:26): And there you you’re potentially missing out of some options, yes. But the restaurant, I don’t know if intentionally or not, they they just put it there and be a little bit paternalistic, as you were saying, and saying, look, you know, this this should be easier for you or the menu of the day or whatnot. Because Kristen Berman (06:41): And I I think if the the best thing is if you go with a friend who’s been there before and knows how to navigate the menu and they could I call it DJing, they could DJ for you and you just set down the menu and they choose for you. And this is actually very freeing for people. We don’t really want to have to live with our decisions of regret. You know, as soon as I pick something, now I have to think about well, if I don’t like that thing, then it’s my fault. And so a lot of decision making is about avoiding the feeling of regret. Kristen Berman (07:09): And this is many times why we just don’t do anything, is because if you have the potential of regret, you’re better off just doing the same thing that you did yesterday, today, and tomorrow. and so I think actually that’s again where gaming kind of gets really nice is the there’s not much downside. You can try a different path, you can try something new, and maybe you’ll lose a life, but you know that you could get one back. Rob Alvarez (07:32): And it’s it’s funny how eventually you you realize that in life there’s many decisions that we think are life or death in the moment and they feel like life or death, right? But they’re really not. It’s like, well just choose something, it’ll be better off than not deciding or going for the default and not doing anything as you were describing.

Forced choice: accept or decline beats continue

Kristen Berman (07:48): Yep. And I think a lot of a a lot of kind of life is this choice architecture. So what are the choices in front of you? And I think we generally forget that not doing something is actually also a choice. And so if you kind of choose not to change jobs, you’re and do the same job you’ve done for X years, you’ve actually chosen that, but it’s not presented to us ever as a choice. And probably if it was presented to you as a choice, we would maybe understand the opportunity cost of these decisions more. In behavioral science, we call this forced choice, where you’re basically giving the user, do you accept or decline something? Kristen Berman (08:19): And when you put that in front of someone, we’ve done one experiment here where if you put accept or decline in front of somebody, people will accept something more than if you just have a continue button. Because all of a sudden the counterfactual of decline is more clear. In a world where you’re just hitting continue, you always have an X button. You always have a get out skip or, you know, don’t don’t continue. And so what people mostly do is nothing. They say, you know, I’m not going to actually continue on this this journey. When you put accept or decline, all of a sudden they realize that actually they don’t want to give up that option. Kristen Berman (08:53): They don’t want to decline something and the cost of saying no becomes higher versus normally the cost of saying no is hidden and people don’t understand the cost of saying no. And so we just do nothing or we exit out or we skip. I don’t know. Hopefully that that makes sense. Yeah. A little bit without without the visuals of the actual design, hard to hard to describe. Rob Alvarez (09:12): Yeah, yeah, for sure, for sure. It it’s it’s a as as as I’m understanding, just to make sure it it’s in the right direction and it may hopefully help somebody if if somebody didn’t get lost in the at some point, it’s you know, say staying at the same job, right? You you never get like a pop up saying, Are you gonna stay at the job? Yes or no. Right. you just continue and you don’t make any choice. When faced with that, you’re really now weighing the options of accepting or declining and you maybe even take more responsibility internally. Like you say, Well, yeah, I actually do want to continue here. Rob Alvarez (09:42): So let me sort of live up to that option. Is that is that part of what you were saying? Kristen Berman (09:46): Yeah, exactly. So if somebody, if a recruiter calls you up every month and says, Here’s another job offer and you have to decide to stay in your job, all of a sudden you realize this is a decision. But if the recruiter doesn’t call you, you forget that you’ve actually still made a decision to stay in your job.

Friction versus motivation in fintech and health

Rob Alvarez (10:02): Yeah, makes sense. Makes sense. And you guys at Irrational Labs have work across, as you mentioned, fintech, health, and many other spans. The stakes of behavioral change in spaces like especially financial and health, as you said, are, you know, the the the the the stakes are real. They’re they they can be immediate, they can be long term, but they’re very, very real. People will feel them immediately. W in in that especially in those kinds of context contexts, what is what is something that designers typically miss? when they’re thinking about creating, you know, their apps or or their experiences for for other people. Kristen Berman (10:34): Yeah, I I I think we always think that people again kind of back to this status quo, I think there’s an assumption that folks will just go through the flow. And instead of kind of narrowly saying every single moment of friction is a potential for them to leave or drop off. Now, sometimes friction is good and it increases my motivation and having sunk costs is wonderful. We’re more committed. But many times up front, if we’re giving people friction, all of a sudden they’re weighing the benefit. Of your app, which they don’t know, they’ve never used, or making an appointment, you know, with a telehealth doctor, or syncing their bank account at a FinTech app. Kristen Berman (11:13): So this they’re they’re weighing this trade-off of what is the friction and what is the benefit at every single moment. And so your job as a designer or product manager is to help someone get over that friction, increase their motivation to do the thing. And so you could do that by reducing. the friction or increasing their motivation. And those are kind of the two levers we tend to play with. Sometimes it’s cognitive friction of having them make a decision versus just this logistical friction of entering a a form or pressing a button. And the motivation could be very surface level, you know, in gamification it’s streaks and points and hearts and gems. Kristen Berman (11:50): In sign up flows, it could be progress bars and congrats, or more deeper motivation about the core value of the product. And even for huge decisions and especially for huge decisions like I’m gonna sign up for a fintech app that’s gonna help me move money or I’m gonna sign up for something that tracks my credit score, we need to the the the underlying motivation people have is still subject to these kind of small frictions. And so that equation becomes even more relevant as the decisions and the kind of the context becomes heavier for folks. Rob Alvarez (12:21): Yeah, that’s that’s basically Fogg’s behavior equals motivation. I’m a trigger and I’m missing I’m missing the other one you just mentioned, B=MAT, skill attitude ap aptitude, I guess is the the A. I forgot for for a minute. Kristen Berman (12:34): In Irrational Labs, we kind of simplify it to these two levers of these two right barriers and benefits. So what are the cognitive and logistical friction, the barriers that people have? And some of this could be their ability, right? Rob Alvarez (12:47): ability. That’s Kristen Berman (12:48): There’s tons of them. You know, cognitive friction can be information aversion, it can be optimism bias, it can be recency bias, it can be all of these things that prevent me from taking an action. And then the motivation is how do I increase the likelihood that I care today? And most of us kind of think about motivation maybe a little bit too far, like you know, kind of think I I have a toddler. And if I were to say, you know, next week you will get the the dessert. Like in no world would that work, right? If I were to say tomorrow you will get the dessert, it has to be immediate. And we’re not toddlers, but we’re not that far off. Rob Alvarez (13:23): Yeah, it’s not so different. And I don’t know i if your experience is similar, but y the ability side or the friction side tends to be what what gets optimized all the time. It’s like always that way. Minimize friction, minimize friction, minimize friction. Yeah, but if I have zero friction for something I just don’t want to do, I’m not gonna do it anyway, right? Kristen Berman (13:41): Correct. Yes. Yeah. So I don’t I think I think these are always a trade. This is always a trade. If you have zero friction and zero motivation, you’re not gonna get someone to do something of lots of friction and zero motivation. If you have, you know, no we need to balance these out for sure. And this is kind of the art of of building products.

The Headspace onboarding experiment setup

Rob Alvarez (13:58): Is there do you have any any any any time where where this this friction something that you can name of course within your with your NDAs and and all this we were discussing before the interview, is there is there a moment where you saw this tension building up and you helped the client navigate through this? Kristen Berman (14:12): Yeah. I’ll you two two examples. one is a experiment we just did with Headspace that’s published. Basically they asked us to to work with them on onboarding. and so to do this, you know, there are lots of constraints. In a perfect world, if I were to try to increase your meditation, I would have everybody around you meditating at a certain time except for you, and you’d be like, wow, I just need to meditate. Sadly, this is a digital world and I can’t do that. And so we’re working within the constraints we’re given. And really what we want to help you do is set an intention that in the future would lock you into that behavior. Kristen Berman (14:44): In behavioral science, we call these kind of Ulysses contracts at the extreme where you’re there’s kind of a a fable Odysseus is you know his lock had his hands tied behind a mast in order not to hear these sirens. And he chose before to tie his hands behind the mast so that if he did hear the sirens, he wouldn’t be able to go and chase the sirens, which Would be would lead to his death. So we’re making a decision up front that kind of locks us into a future behavior. we can call this pre-commitment or whatnot. And so for us, the closest thing in Headspace we could do to pre-commitment was having people sign up for a course because they’re basically getting signed up for a course, and then in theory, they would take the course every day. Kristen Berman (15:25): Now, this is clearly not a a a full commitment contract because we can’t force you to take the course. It is something that mirrors the real world, right? Many times people will go to these 10-day meditation retreats. And the day one you walk in, it would be hard for you to walk out. It’s possible people do, but you’ve locked yourself into something, said, I have an intention to do this, even if it gets hard, you’re probably gonna sit through it because you’ve already paid and everyone else is there. And so in a perfect world we’re having people set their intention up front and then follow through at the end. Kristen Berman (15:56): So what we did in the signup flow is that, well, how do we get people this to sign up for a course? We can’t just default them into it, otherwise the whole thing is it’s not not that interesting. And so we had multiple versions of this signup flow. And in one case, we asked them questions. We say, look, tell us when you want to meditate, tell us your experience level, and then we recommend them a course. And we had two versions of that condition. In one condition, we just always would recommend the same course, basics, regardless of their answer. And we didn’t call it basics. We called it something fancier than that. Kristen Berman (16:28): But the other condition was clear personalization. You’re gonna say you want to help with sleep, and so we’ll recommend a sleep course for you. And another condition, we actually just gave them the basics course. We said, look, we actually do believe that this course works for most people. We’re not gonna ask you all these questions. We’re gonna make a recommendation for you. They’ve already done some sign up flow questions in the, you know, in the kind of pre pre section, we’ll make a recommendation on basics, but everyone’s getting it without kind of answering these questions. And so kind of any guesses at which condition drove the most course signups. Rob Alvarez (17:00): It would be interesting because one, the the one that’s quote unquote personalized feels like I have more ownership over it. So that that sounds like I would be more motivated towards it because it it feels like I’m already invested there, right? However, the other one’s a bit more paternalistic and we were discussing that before. So I have I have recency bias on that one right now. So I’m I’m really, you know, I’m I’m I’m not not sure. My default answer would have probably been the first one, but you know, these things can turn out many ways. There’s many, many different theories Kristen Berman (17:30): I mean, There are and I think this is kind of one of the key lessons in in any kind of design is is it’s really hard to drag and drop one answer to another. So just because friction worked for you before doesn’t mean it’s gonna work for you in a different context. We have to be more thoughtful about the the theory of change.

Non-personalized questions doubled course enrollments

Kristen Berman (17:45): So the thing that worked the most in actually doubled course enrollments was giving them questions that was not personalized to the answer. So I’m just going to give you a set of questions and then I’m gonna recommend the same course that everybody gets to you. There’s no personalization. So if we think about why that worked, so one, we can say we’re gonna compare it to the the no question condition. So do we have questions or no questions? Because they’re also getting this like the same course recommendation. And so the theory there would be actually like you’re just getting people invested in the answer. You can ask them whatever questions you want and you’re getting them invested in the answer. This is kind of this sunk cost. Kristen Berman (18:21): Adding friction is not a bad thing. Of course, if you add really hard questions where you’re asking them to type out a lot, this I don’t hold to that, but In general, when you’re asking people questions, they may get more committed to the answer. The second finding is that it’s we’re always solving, I think, as products and marketing folks for the upside of an answer. Could I get the best course recommendation for you? So imagine like I perfectly understand you and I know that you’re really struggling with sleep. I’m gonna get the best course recommendation possible for you. And it’s gonna be a specific sleep thing based on your level of sleep issue. Kristen Berman (18:55): And what we really should be solving for is not getting it wrong. Because in personalization, if you get it wrong and let’s say you care about sleep, but I’m gonna give you something about managing anxiety that’s not sleep, now all of a sudden I’ve just lost you. And so the idea of kind of personalization works if you probably nail it, but also if you don’t nail it, people are bouncing. They’re not interested. This is not the course for them. And unfortunately it’s really hard to fully understand you as a human with a few questions. So the likelihood that we miss is is real. Like I I’m just not going to all the time get it. Kristen Berman (19:27): And even if I did perfectly predict it, maybe you have a, you know, a a a different way that you like to be talked to. And we didn’t didn’t answer that. And there’s no way to understand that from a set of questions. So I think personalization gets a little bit more of a kind of love from the product community than it deserves in some of these cases because the downside effect is just just so vague.

Why more enrollments did not mean more meditation

Kristen Berman (19:46): The one thing I would say is we got doubled course enrollments from this intervention, very impressive. And we did not get more people to meditate. Okay, so let’s so so so we’re gonna if i i if you’re okay, I can can I unpack this for a bit? Rob Alvarez (19:59): What why Please, please go ahead. Kristen Berman (20:01): great. So so one, the in behavioral science, this idea that I can motivate you in the moment for something is very clear. Like if I were to have a very clear behavior of getting you to tie your shoes, I could design a system very easily to get you to tie your shoes. Now, could that same system get you to tie your shoes tomorrow? Probably not. Could it get you to tie your shoes the next day? Probably not. You need another system to get me to do that. And so I think in behavioral science in general, we kind of think about motivation as if it’s going to forever la or many times people think about like it forever lasts. Kristen Berman (20:36): And in behavioral science, we think we actually get to motivate you in the moment. So what we didn’t get to do with Headspace was take over any of the screens when they came back into the app. All of a sudden they have this course they signed up for, maybe they signed up for it two days ago, and it’s at the bottom of a list. Or is you know, it’s like hidden in kind of the tabs of Headspace’s machinery. It’s not prominent on the today tab. And so I think that that in itself is a lesson. So one we should have fought for that. We maybe we should have said we can’t release it without it. Kristen Berman (21:08): Unfortunately, that’s sometimes not how how things work. And so I think the the the the step by step lesson though is you can fix some part of the funnel, either your behavior change or your user’s behavior change, but you still have to focus on the next part of the funnel. to drive kind of that repeat repeat cycle. Rob Alvarez (21:24): Yeah. Makes sense. Makes complete sense. Of course if you if the intervention lasts here, here’s where you get the impact. How is that gonna roll out to the rest of the the year, the rest of the world and everything else? That’s a lot harder to predict. It’s just like with, you know, when you have in statistics, right? You have these models, you know, predicting tomorrow’s purchase is gonna be always more accurate than predicting next ten years from now. You know, you like so many things can happen in between for sure. I Kristen Berman (21:52): Meditation meditation is very, very difficult. So if we think about kind of like one of the hardest behaviors to change, you do it once and you don’t see that much many benefits. In fact, it’s actually the hardest it will be the first time that you do it. And so getting someone to return is the whole thing, is the game. But you have to get them to start first. And so kind of there’s these two pieces of like breaking, you know, breaking the seal and getting someone to to start and try, but that’s also going to be probably their their worst experience that they have on the app.

Pre-commitment, lost streaks and why willpower fails

Rob Alvarez (22:21): Yeah, for sure. For sure. many things came to my mind. Just w the first one is I think it was you who posted about the Arnold Schwarzenegger thing that he he gives people like a fine for not committing. So this is this pre commitment as well. And it’s painful. And it it it it’s a system that works in a in a way, quote unquote, every day, because you have to commit to those days at least, so that you get your money back. And you’re always thinking about the money, the money, the money. It’s always there, right? It was you, right? That I saw it on Kristen Berman (22:49): Yeah, you’re giving re people a reason to do something today. So so today, do I have a reason to work out? It’s like, well, I have a reason to do all my emails because people are yelling at me that I have a reason to take care of my kid because I have to pick them up for daycare. Do I have a reason to work out? And you know, th th this is just giving us it’s manifesting ways in your life to make that a top priority versus level seven priority, right? You’re you’re giving yourself a reason to do a thing today. Rob Alvarez (23:14): Yeah. Yeah, for sure. And then then then the other thing you were talking about meditating. I’ve I’ve I I used another app. I s actually still have it. And I got to meditate for, I don’t know, year and a half, almost every single day. And it had a streak system, which I talk about in many different ways in other episodes. And suddenly I lost my, you know, I don’t know, eight six hundred day streak or something ridiculous like that. And even though I I did see the benefits, I’m still convinced of how important meditation is. I like to meditate, actually. I haven’t meditated consistently over the past, I don’t know, year and a half after that. Rob Alvarez (23:49): Streak mechanics and all that. It was just remembering how hard it is on meditation, even when you understand and you know the benefits, to keep up with it in daily life, e even reducing the barriers, increased ability, like triggers, everything is there. And still, you know, it’s not an easy behavior. Even being a behavior design, you know, being in behavior design, knowing all the like all this the stack was there and you know, I just I just lost it. And it lost a streak amongst other things, which yeah. Kristen Berman (24:16): I and and I think a lot of meditation kind of represent this at the very very clear level, but it is a thing that we do alone by ourselves that no one really no one cares about except for us. Really, maybe you have a spouse that’s trying to track you. In no other world in kind of a work or professional context would that happen. So imagine I hire you and you’re onboarded. I would check in with you. I probably won’t check in with you daily, but I would definitely check in with you probably your first week multiple times, ask you how things are going. Give you a goal, see the deliverable. Kristen Berman (24:47): You would have a meeting, we talk about performance. We basically have a system by which, by the way, almost across all cultures, this is how work works, right? You have presentations and deadlines and KPIs. And yet when we do our personal habits, we kind of just throw that out the window and say, I hope that I do well. I’m not gonna tell anyone, I’m not gonna set a goal, I’m not gonna have an incentive, I’m just gonna hope that my willpower is enough. And it turns out it’s just not. And so we really, if you want to succeed in any of these things, have to go to the extreme and kind of create that system for yourself. Kristen Berman (25:19): Otherwise it’s just even if you know all the the tricks, it’s just hop hopelessly impossible.

AI personalization and the health meta-analysis

Rob Alvarez (25:24): Yeah. Yeah, for sure. For sure. Absolutely. Kristen, you also mentioned personalization and you and you got into that. And that was one of the things I was I was thinking of because t nowadays with AI, right? now AI personalization is obvious, right? Everything needs to be personalized. And apparently or they say that AI is very useful for For personalizing. I I don’t agree with what I just said entirely. In fact, I don’t mostly don’t agree, but that’s that’s the general feeling. And people talk about personalization at scale, right? But it’s it’s at least in my experience for what I see, that personalization is is not considering everything else other than the technical side of it. I don’t know what you’ve seen in in in your experience nowadays with with AI and personalization, behavioral science, how that’s being connected in that in that world. Rob Alvarez (26:09): What what have you seen? What do you think about? how AI is personalizing and how useful it it’s getting to be at this in in that particular area. I know AI is useful for many, many things. Kristen Berman (26:18): Yeah. So I I think I’ll I’ll say a couple things on personalization. So one is again, this getting it wrong is is really tough. So if you’re building a product in an international sense, you want to quote unquote personalize for people. You don’t want people in Brazil to see the same thing as people in Germany. And that is kind of clear. And and I would think about personalization like this. Like like you want to get the the big things right if they’re returning versus not returning user. They’re an expert versus not an expert. I would say international is a big is a big one as well. Kristen Berman (26:50): If they are a freelancer versus work at a large company. So there are large kind of personalization categories that can be helpful. I and and are helpful specifically, especially if you get it wrong and you show something to somebody that they they don’t expect. I think the AI personalization implies that there’s a much more nuanced view. That we can deliver to people that would really get at either their psychology or their core job to be done. And I d I don’t know. I think the verdict’s out with this. In in health, and this was kind of a study or or meta-analysis done around 2020. So six years ago before AI, there was no so it’s meta-analysis and meta-analysis found no effect of personalization as it comes to health advice. Kristen Berman (27:36): So imagine I give you a score, you know, you’re Age, your real age, or I give you some score about your health, in theory, this should motivate me to take some action. In this meta-analysis, they found that across all of these behaviors, from smoking to physical activity, that it did not drive this type of behavior change that we would expect. So, in that said, all the hows matter. So how I deliver that information to you is important. If, you know, if you’re somebody who has a if if you’re just kind of doing a simplistic UX, this may not work. Maybe if you’re more evolved and you have something that is using social proof and using anchoring and there are things that could drive upside more. Kristen Berman (28:15): So whenever we hear it doesn’t work, we always think, well, how did they design it? Yeah. But I think we should remain skeptical about the upside from personalization if you’re forgetting the table stakes of how humans behave, right? So in general, we kind of have core principles of human decision making. And so we should first rely on those. And then we could say we could also sprinkle some fairy dust of personalization onto it to drive gains on top of this table stake type of type of finding. By the way,

Reminders, Duolingo timing and Netflix ratings

Kristen Berman (28:44): this is also what I would say with with messaging and reminders. If people have done reminders, it’s very hard to get an effect on a different text of your reminder. So sending reminders tends to get an effect. The idea that I’m not going to talk to you at all versus I’m going to talk to you is interesting. The frequency of reminders tends to get an effect. What doesn’t tend to get an effect is the slight text changes to that reminder. You can find the effects, but they’re much smaller and harder to find than just the base level table stakes. Do you send a reminder and how frequent is it? Kristen Berman (29:16): And that’s what I mean by table stakes is kind of the core of the product. And then yes, we can customize for folks, but that’s going to be more incremental than than the big stuff of just nailing kind of the core, core elements of decision making. Rob Alvarez (29:29): You’re you’re talking about that. And I’m I’ve been recently been looking into Uber Eats and one of the things that they apparently get very well. I’ve I’ve been starting to to study what they what they do and the views of other people as well, is the timing of their notifications and how they randomly give you different timings for notifications to maybe find you at the right time. And that seems to fit what you’re saying, but also also opens a new a new gate in a way. It’s like, well, do you send reminders? Yeah, how often? Yeah, once a day. Not just once a day. For for that specific product, of course, because meditation would not maybe not make sense. Rob Alvarez (30:02): For example, we were just talking about it to remind you at six AM and then at ten PM in a different day because in theory it’s you know, it should be a habit. It’s make makes it easier if you do it at the same time. But but Uber Eats is different. You can order food at different times of the day as long as it’s eating hours. Is is there is that does that ring any bells for you when I when I say that? Kristen Berman (30:23): Yeah. So Duolingo has has come out and said that their magic recipe for reminders is sending at the same time of the day that you signed up. So if I’m signing up at Tuesday at 10 AM, that’s a signal to them that at Tuesday at ten at ten AM I’m thinking about learning a language. So the next time that they send me a reminder, it would be that time. And they’re drafting off of my past behavior. And I think this is a general lesson is like the thing that I did yesterday is a thing that I’m likely to do today. And so it may seem random to you, but it also may be cues of like, you’re opening it up at lunchtime a week ago. Kristen Berman (30:58): We’re gonna take that cue is that you are open to delivery around lunchtime. And maybe if you open it up at eleven to check, but you didn’t order ’til twelve, they’ll take cue is actually at eleven is when we should send the reminder. So I think in general companies have tended to learn that too is it is, and this is kind of a Netflix finding as well, that their algorithm improved when they had people rate more movies. So in one world you could say, look, I’m going to just draft off of what most people like or most people like Kristen like. So if we know that she has some characteristics, I’ll find the person that’s most like Kristen, I’ll create an algorithm based on that. Kristen Berman (31:35): Turns out that is incredibly difficult. And the thing that’s much easier is to ask me about past movies that I’ve watched and then draft off that. And use that as an in is to it to create the algorithm. And so kind of the finding there is that when they made it easier for me to rate movies, they went from a five-star to a thumbs up, thumbs down. More people rated movies, which meant their algorithm got better because more people were giving them signal about the movies that they liked or didn’t like. So as a kind of core principle, past behavior predicts future, you can say I understand somebody if I can understand what they did yesterday or the day before.

Where to find Kristen Berman

Rob Alvarez (32:08): Amazing. That makes a lot of sense. Thanks for that, Kristen. And you know where I we don’t wanna take all your morning, I guess, where where you’re at. Where can we find out more about Irrational Labs, your work, or anywhere else you wanna lead us? Kristen Berman (32:22): So the place to go to be irrationallabs.com to understand some of these case studies that I I mentioned just now. And then I also have a Substack called Product Teardowns where I’ll do a five-minute teardown of a popular app and ha try to walk people through how a behavioral scientist thinks about design and user engagement, pricing, incentives, loyalty, kind of in the very tactical level. So you’re gonna like go into the UI with me and watch as we explore the product together. Rob Alvarez (32:51): Amazing. Thanks very much for that. We’ll put it in the show notes. However, Kristen and Engagers, as you know, at least for now and for today, it is time to say that it’s game over. End of transcription

Discover more from Professor Game

Subscribe to get the latest posts sent to your email.