Episode Transcript
Speaker 0 00:00:01 If you are leading growth, building a startup or looking to ladder up your skills, then you're probably really busy. So every other week tune in to growth snack, the breakout growth podcast, short where Ethan Gar and I share one key learning to help you on your journey to break out growth, success. It's food for thought for anyone hungry for growth. Alrighty. Then let's talk about the concept of write the report before you run the experiment. What is it and why is it important?
Speaker 1 00:00:28 It's a really simple concept. My hypothesis is that if you're leading growth and you want to run better experiments more frequently with more predictable outcomes, then you should work to build a habit around writing the report before you run the experiment. And why is that important? Well, I think we're all driven to run high velocity testing. We want to run more and more tests so we can out test our competitors and get those learnings. But it's only valuable if it actually produces quality learnings that are going to power, that flywheel where you take a learning and you apply it to what you're going to do next. Um, so if you write that report before you run the experiment, I think you're going to improve the odds that you're going to get those actionable insights that are going to matter and help them grow.
Speaker 0 00:01:08 Right. Okay. So what happens if you don't do this?
Speaker 1 00:01:10 Yeah. The danger is that instead of your experiments, turning guesses into facts, you're going to turn guesses into either more guesses or more unanswered questions and actually think it can be really a culture killing the state. If you do that. Um, while we, again, we're all trying to run these high velocity experiments. When your growth meetings turn into these unanswered questions where you're, everyone's pointing at each other saying, did we learn anything? Do we do, did we get the data? Does anyone understand what we learned? It really, it sets you back and you might think you're achieving something. Cause you're running a lot of tests, but they're not really experiments if you don't get anything out of them.
Speaker 0 00:01:46 Right? Yeah. I've, I've definitely seen teams, particularly teams that go from not doing any testing. They suddenly start testing to follow my advice, to hold yourself accountable on how much testing you're doing. I really pushed high tempo testing for a long time. But what happens is that they start to just go through the motions of testing. They're like, okay, this is what we're supposed to do and almost become robotic around it. And uh, I think when you do that, you're just, you're not learning what you should be trying to learn from that testing.
Speaker 1 00:02:17 I mean, I definitely blame you. This is definitely all your fault, but no, I mean, uh, truthfully, you know, you got me super excited about this idea of high tempo testing. I heard it from you, uh, you know, early on, I think maybe while you were still writing your book with Morgan brown and right away, I took it. I went back to my, my team and I was like, we're going to do this. And it got everyone excited, but we got ahead of our skis and I take full credit and blame for it. Um, we, uh, we kept making these unforced errors, you know, because we would run what we were calling an experiment, but we didn't have a fleshed out hypothesis or we didn't have the data. Or even if we had the data, we didn't have the skillset to actually understand that data.
Speaker 1 00:02:56 And it really just, it was, you know, I think we're in of really setting ourselves back instead of moving ourselves forward. And I remember I was chatting with our director of engineering at the time and he was telling me about test-driven development, which is this idea. And I honestly I've stole a lot of this from, from this idea of test driven development, where you write your test case before you write your code and you do that because it has these forcing functions. It forces you to make sure that you're just writing code that meets the requirements instead of adding stuff on top of it, that's irrelevant or writing stuff. That's just irrelevant because you got excited about something else. Um, and it also has this neat forcing function of getting teams, talking to each other people, you know, talking, why are we doing this? Why does it matter? And you know, so I just sort of stole that and turn this into this thing. Thought on report driven testing where you write the, the report before you write the experiment. And I, as I piloted it with teams, I've really seen it make an impact.
Speaker 0 00:03:50 Yeah. You know, this is really consistent with, uh, something we teach in the go practice simulator, which is when you're creating an experiment and you're building an experiment document. One of the key components of that document is what are we going to do based on the results of this experiment and it's, and it's often one of the most overlooked, uh, areas of an experiment document. And it's something I really learned from Oleg and, and have put into practice and seeing the value of that. But it, it actually reminds me of, uh, one of my favorite books over the years, uh, seven habits of highly effective people from Stephen Covey, where he talks about begin with the end in mind. And so I think if you're really intentional about what you're going to do, based on the learning from a test, you're much more likely to run that test in a way that you're going to get that learning. And the learning really comes from the report. So this fits really nicely with that concept of, of, you know, what, what is the outcome that we're shooting for, what we're going to, what are we going to do based on that outcome? And let's make sure that we know exactly what we're going to use in terms of information to make those decisions. Yeah,
Speaker 1 00:05:02 I think Ali is really good about sort of running that sort of, you know, helping you understand how a systematic approach to this can really drive that end goal more efficiently and more effectively. I just, one of the things I really loved about go practice, but I, you know, I think with that too, is when you, when you start with that end in mind and you know where you're going, it just makes the whole process easier. I mean, if you think about it, like you and I are very anti-hero bureaucracy, we're not, we don't, I don't want people to write extra reports and extra documentation. You're going to have to do this anyway. But if you read this earlier on the process, it improves the odds that you're going to succeed. And look, I think, you know, you pointed out the spec and I mentioned, you know, early on for me with, uh, when I was leading a growth team, it was really important at the beginning.
Speaker 1 00:05:47 And I think for most teams, this is going to be just good growth hygiene. It'll help you, um, just run more effective experimentation. But obviously if you have a really highly tuned experimentation process, I mean, if you're carrying Flanagan at HubSpot or Morgan, uh, brown, your co co-author of hacking growth at Shopify, you know, we've talked to them, they have incredible tools available to them to do just awesome experimentation. If they don't do this ahead of time, they might be okay because they have those tools. They have all the infrastructure to support their experiments. But especially if you're getting going in this, I, you know, I think this is just one of those simple things you can do to really improve your odds of success.
Speaker 0 00:06:29 Yeah. So this is, this is really a quick introduction to this topic. And, and, uh, as we, as we talk about with growth snack, we're really trying to have this be food for thought that that gets the wheels turning so that you, you start to look for ways to improve how you and your company are approaching growth. But if somebody wants to go deeper and really dig deeper into this topic and, and think about specifically, how, how do I apply this? Where can they go for that information?
Speaker 1 00:06:57 Yeah. I mean, I've put a lot of thought around this and I actually wrote a short article on our shared website, breakout growth.net. Uh, it's called write the, write the report before you run the experiment. So easy to find if you go to our blog. Um, but yeah, just in general, I just want to let our audience know that, you know, we created breakout growth.net for them. It's a place where you and I share any thoughts we have that we think will help teams accelerate their growth trajectory. So, um, certainly, uh, check it out there. There's a lot more, it's a short article, but there's a lot more depth to this. And if you have any questions don't hesitate to reach
Speaker 0 00:07:28 That's me. Absolutely. So that's our first, uh, gross snack. Um, this is something that, uh, is, is something we're hoping to do on a regular basis as part of the breakout growth, uh, podcast, but just a short version of it. Um, I consider it a pilot. So, um, we really want people's feedback on that. And, uh, if, if it's positive, that's great. We'll keep doing it. If, if you don't find it valuable, um, I think that's important as well. Um, so you can reach out to us through Twitter or LinkedIn, send us a message, or even through the breakout growth.net site, there's a link on there to contact us. Um, and, uh, yeah, if the feedback's good, then we'll, we'll start doing this every other week in between our, our longer episodes. And, uh, and you know, even as you, as you reach out, there might be some specific areas that you'd like us to cover. So let us know that in the messages as well, but thank you everybody for tuning in. Yeah.
Speaker 1 00:08:26 Let's talk about gross. Made me hungry. Let's let's set it.
