👈 Home

Tim Duke on SaaS Experiments


November 2020

In this episode, I talk to Tim Duke, Director of Operations at CRO agency Mobile1st.

We talk about different kinds of CTAs - chats, forms, phone calls - and how to reach the optimal balance of all three.

You can listen on Apple Podcasts, or Stitcher, or right here 👇

Listen on SoundCloud

SaaS Experiments · Tim Duke on SaaS Experiments

… or on YouTube

Quotes

Transcript

Brian:
In that magical moment when a visitor to your website decides that they want to talk to your sales team, what should you offer them? The chat widget form to fill out or just a phone number to call? Well, the short answer is yes, and the full answer is it’s complicated, but lucky for you. I talked to Tim Duke from mobile first, and in this episode, he’s going to unpack the whole topic.

Brian:
Tim Duke is the director of operations at mobile first and Austin based conversion rate optimization agency who works with clients like flex seal and Erin Condron design. But today he’s here to tell us a story from his days at a cloud hosting provider. Tim, thanks for coming on the show.

Tim:
Glad to be here, Brian.

Brian:
So let’s go right into it. Tell us about the company and the product that you were working on.

Tim:
Sure. So this was a billion dollar plus cloud hosting provider and I ran their CRO programs. They had major major products that I think there’s probably 50 different websites that were involved, but every single website or every single product, depending on what lens you wanted to look through, it had its own set of AB testing priorities and ways to get new business. Does that make sense?

Brian:
It makes sense. It sounds a little overwhelming, I guess

Tim:
That is exactly how I felt for several years.

Brian:
Okay. Well, we’ll tell us more and I guess I won’t ask you to list all 50 products and KPIs and programs, but broadly. What, what kind of KPIs were you tracking here?

Tim:
Well, it all came down to leads. It was B2B. So you get business through what I would consider the three big channels and that’s phone calls, emails, or leads or chat conversations and all of the 50 different sub domains and the different products. That’s how they all generated their conversations. And so it was a question of like, how do we run a testing program for this product? Because there’s one product might make $150 million a year. And if you could get a winning result out of an AB test for that particular product, you would end up with a pretty good return on that investment for the test you ran, the question became like, how do you blend those tests together across several different products and prioritize your three major CTAs?

Brian:
So let’s get into the weeds with it and talk more about some specific tests in this sort of chats versus forms versus calls, theme, or

Tim:
Saga. The saga starts with how the organization was structured to, we need more leads and a lead could be a form. It could be a call. It could be a chat, whatever it is like you want to start more conversations, so let’s go get more conversations. And so we start up with, you know, the basic testing of, instead of your CTA saying, contact us, let’s do a CTA that says request a free quote. Okay. And three months later after the 95 different stakeholders agreed that it was a concept that they could get behind. We were able to run that test and to no one’s surprise, at least from within the CRO community request, free quote got more clicks and it got more engagements and it started more conversations and it created more leads as opposed to just the, the very bland, no inspiring words, contact us.

Tim:
Sure. So that was the first little, Oh, we’re onto something here. And that’s when all of the different product managers and different parts of the different sites got on board with like, okay, we really understand that if we change things on the site, user behavior changes and we can give more stuff, let’s do more of that. Nice. And yeah, that was a big win. It was one of those, like I felt bad for testing something so obvious because if you throw free into your CTA in all caps, it’s going to get more attention, but it was one of those I would consider it like a culture education test to let people know, Hey, this is how this can work. And we’re not chiseling things in stone. It doesn’t have to be your idea versus my idea. Like let’s just test them all.

Tim:
But that’s where it got a little bit more complicated. We got that first big win and we saw more engagements, but then where it took a twist was figuring out that the enterprise, not only did they have different products with different product owners, they had a different sales structure for people who ran the chats and people who responded to emails and people who answered the phone and each one of those teams had their own quota. And so we had to run a test and I say, had to run a test because somebody, several rungs higher in the org chart than me decided that, well, I’ve been told we need more chats. Chats are, are where good leads are coming from now. So let’s go get more chats. So we had to run a test where request free quote, instead of when you clicked on that button, it pulling up a form where you would request a quote and put in your name.

Tim:
And this is the sort of infrastructure you’re looking for. You would click on request free quote, and it would pull up a chat box, which is a bit of a bait and switch, but the marching orders were, let’s get more chats. I can get you more chats, just change all the buttons to click chat. It’s not going to be a high-quality interaction. It’s going to confuse a lot of people. The conventional experience for when you ask for a quote is that you’re going to have to fill in some information, not start talking to a robot or somebody behind the keyboard. But it was yet another enterprise political struggle of, well, we need more chats, do it at any cost. And so we ran the tests and obviously this click-through rate for request free quote stayed the same because people didn’t know what would happen.

Tim:
They all clicked on request free quote, but half of the audience got a chatbox that they immediately closed. So not only did chats not go up because we were basically baiting and switching people and to try trying to force a chat conversation. But we got rid of a bunch of form fills and leads because we took that opportunity away from them. And so it was, I would say probably a 60 day exercise on just because you want your users to fill in a form or to start chatting. It doesn’t mean you can make them, you have to think in terms of how does your user or your prospect want to engage with you. And for this particular hosting company, we were talking to a wide range of potential customers. Some are at the C level and they wanted a quick just get me on the phone and let’s go.

Tim:
And then somewhere at the dev level or entry level or even intern level of, I was told to go find a new hosting thing. I’m doing some research. And so they were more comfortable with chats and they were more comfortable filling in a lead and setting up a call later. It became clear that it’s not about which is better phone calls, chats, or forms. It was what type of user is ready to fill out that type of engagement. We can’t just make everybody chat right now. It’s just not going to work for a lot of different scenarios and a lot of types of buyers. So we ended up at a let’s do a three-card Monte sort of style of all CTS are set up with the same visual priority and expectation. You’ve got three buttons in front of you choose the one that you want to interact with, make it clear like this one is for phone calls.

Tim:
This one is for emails. And this one is for chatting with people right now, let the user decide what they want to do. That turned into a really insight because you’re suddenly not forcing users to take the action that your quotas mandate, you can get to a higher quality scenario. If you present the options that are available and let it use your decide, just getting to that level of, okay, what are we good at? Well, we’re good at all of these. We’ve got chats, forums and calls, and all of them are great. They come in at different volumes, but let’s recognize that we can’t really just remove one of them. So it, it took, I think the conversation went all the way to the C-suite to talk about the idea that we put our CTA CTS up on a platter and say, okay, let users decide.

Tim:
And it was a big risk and it took two or three months before we opened up the idea and got permission to even run a test, to think about how we might possibly consider going into wait a second. Why does the phone team have the same real estate on the page that the chat team does and ran the test and saw that everybody had an increase in chats, leads and forms leads went up across the board really? And it was one of those aha moments where we’re no longer trying to cannibalize another product team or another sales team’s numbers so that our numbers go up. It was, Oh, some people want to chat. Some people want to call, some people want to fill out an email. Let’s just do that. I don’t know why it took me two or three years to get to that level of political capital to run that sort of test.

Tim:
But it also helped the organization understand that they’re incentivizing their sales team. And they were thinking about their products and a bit of a backwards way because their sales team was thinking about just more calls or just more chats at any cost. And their products team was just thinking about, I need more leads, get me more leads. And those two opposing structures of, I need more leads regardless of what channel they come through and a phone team or a chat or a email team saying like, no, no, no, no. My emails went down and never looking further downstream to, okay. Emails went down, but how did that change? Our overall mix of leads? Did we get more leads? Are they higher quality? Did they get more valuable? You’re incentivizing your sales team for the wrong reasons or in the wrong way, right. Somewhat of a, an afterthought.

Tim:
Now I remember that we ran a test where we were focused on increasing chats, you know, with a little slider that goes in and out create more interactions with the slider. And so we got chats to go up and we were running it and comparing it to the leads with the emails. And then we found out 90 days later that the email team and the chat team were the same people. And they had just stopped checking the email leads because they were getting so many chats. Wow. So we shot ourselves in the foot in the sense that email leads were really valuable, their, their, their average purchase price and time to close, which so much faster than we just didn’t check the inbox for several months. And that only became clear from the marketing ops team several months down the road that said, Hey guys, what, why aren’t we selling anything anymore?

Tim:
So I think the thing that I’m circling around here is the biggest, most important thing you can think about with what CDA to choose, whether it’s forums, calls, chats, isn’t which one’s better. It’s what are you really good at? Like, what is your org set up to do that can be successful if your org is set up to answer the phone at the drop of a dime, and everything’s fine focus on increasing phone calls. If your org is set up so that your chat team fights with your email team or no one answers the phone, think about how that should go into your digital strategy. The biggest conflict that I dealt with wasn’t which CTA was best. It was what’s behind this ETA that matters more. Are we really good at chat conversations? Can we close the deal faster on chat than we can on phone cater to your strengths and then focus on your time to close from those channels, know what you’re good at?

Tim:
And don’t try and make your product team fight with your sales team, for what should be the same goal. We want to create more valuable conversations. Don’t worry about how those conversations happen. And I think that that’s the shortsighted scenario of who’s in charge of the thing that makes the money. And that’s where the decision needs to really be made, not at a particular interaction or channel level, but at the very top where it says, okay, like we’re going to accept that the phone team might not be busy for the next month because we’re trying to figure it out. If we can create more conversations through chat and then weighing that out, it’s a long game. And it takes a while, especially in B2B, especially on very large, you know, several thousand dollars, hundreds of thousands of dollars per month items like that sales cycle is going to take awhile. So you’re not going to be able to run a two week test and understand what your impact is. You’re running the test for 30 days, you’re running the test for 90 days. And then you’re looking at the quality of those leads over the next 90 days. So you’ve got this sort of leapfrog pattern of did we set up our tracking properly? How valuable were those leads during that test?

Brian:
We started off talking about the company and the website and how they make money. And the basic conclusion was more leads. Is the goal get more leads. That’s how we win. But the reality of how to win sounds like with time you realized was, was much more complicated than that.

Tim:
Yeah, I think the biggest eye-opening experience in terms of testing and how to run the right experiments, the reality is accepting what you’re good at and not chasing glitter or setting your priorities that are competing with other people within the organization. You can’t have get more leads, but also get more chats, but also get more forms. If everything is the most important priority, then nothing’s important. So that comes down to a strategic decision on how you build out your org and how you set your team’s priorities more than it. It’s like, how big should the chat button be? Or should we use a chat robot versus an automated phone call? So I think the most important thing to think about for CRO and improving your B2B funnel is really at what level within your organization, are you allowing somebody to choose the priority? If everyone has authority than chaos reigns, which is somewhat defeatist.

Tim:
But I say that is as someone who’s spent almost 15 years on the receiving end of a conflict on teams working in opposing directions, right? So the biggest lesson I’ve I’ve learned so far is that you have to have somebody at the right level to say, okay, I recognize your goals. And this team, they might not be met this month because we’re going to run a test and I’m not going to hold you this team accountable for that goal. We set because we’re testing this and it should be okay for us to test this. We can’t all have these lofty goals and not take some risks.

Brian:
So you talk teams with different priorities or competing goals. And it sounds in a way like CRO was almost in the middle of that, at least in terms of how the leads were coming through the website. If you were talking to somebody who was new to an organization in a CRO role, and let’s say that the website is chats and forms and calls, and there’s a chats team and a forms team and a calls team. Do you have any advice for how to go about feeling out these tensions or priorities or how best to orchestrate all of these teams so that you don’t cannibalize each other?

Tim:
I think it comes down to building the trust with each one of those teams or those those stakeholders, but then leveling up a couple of rungs above either marketing ops or sales ops to get buy-in to say, okay, VP of sales, are you comfortable? And can you, can you back me up so that we can run a test that maybe gets rid of phone calls for a month and don’t hold the phone call team accountable for their lower numbers. You need a super senior level person to say, okay, we’re just going to do this. And I don’t want it to affect people’s paychecks and their incentives. And I don’t want them to feel like they’ve done the wrong thing. We’re trying something out. You need testing buy-in at the highest level. And from there, you can start to really calibrate, where are the most valuable leads coming from and how do I change the structure of my org to better suit our potential buyers.

Brian:
So overall did this experience and this sequence of tests and these complicated internal organizational results change your approach or how you think about optimization and UX.

Tim:
I think it changed how I think about where optimization needs to live within an organization. And at what level you get your political support. I spent quite a bit of time trying to help great people on great product teams, do things that help their numbers. And it took a lot of hard lessons to realize I was doing that at the detriment of other teams. I was increasing the chat teams and they love me, but the phone call team stopped loving me. And that’s when I that’s when I realized that it’s not really about that internal org priority, it’s about the user priority. And certain people are going to chat. Certain people are going to call and just giving them the option and thinking about the holistic picture is the most important part. So that way you’re not thinking about, well, we need to make this button bigger.

Tim:
We need to make the phone call just automatically dial whenever you show up on the homepage, whatever. And so my biggest lesson was really that optimization needs a seat at the biggest table. That way they have the authority to pull on the biggest levers, because at the end of the day, optimization is something that has to spend several different teams and several different roadmaps. And you need buy-in. And if you go around trying to beg barter and steal from each little team, you’re never going to get there. It’s gotta be a top of the org decision to have that level of risk mixed in. So when I see teams that are companies that say they’re entirely sales driven, you sort of walk away knowing like, Oh, okay, so we’ll do anything. We’ll sacrifice the product to close the deal, the same sort of structure of, okay, well, if your company is incentivized by certain areas or are there, it’s set up by channel and they’re incentivized by channel, then your channels are going to fight each other and that doesn’t move the needle forward.

Tim:
It just helps one roadmap versus the other. And the biggest thing that I’ve learned is is if your org set up right, and you’ve got the right buy-in at the, at the senior level, you’re going to end up helping your users more because you’re creating an experience that is more suited to your different types of buyers, your people on the subway that just want a quick answer. The people that are driving home and don’t want to mess with the form, they’re going to dial the phone and they’re all there. Like people do different things everywhere, always. So give them the option, work to solve your user’s problems first, rather than your organization’s goals.

Brian:
So to me, the biggest takeaway here is that you don’t want to force your users into a single channel because it fits your organizational structure. You want to listen to your users and offer them the channels that they want, and if anything, reform your organization to better meet those needs. So in light of that perspective, I wonder what your stance is on an organization that is currently in the process of deciding we need a chat platform, we need some sort of automated chat. We’re going to add it to the website. How do you think about that?

Tim:
So that’s something that’s come up several times over the last several years. And and some scenarios, it got really difficult because it was, you know, senior level person saying we needed automated chat. And there’s a dozen people that are typing in chat boxes sitting next to them. So well, so we want to get rid of all these people, is that, is that a comfortable position? Are we good with that? Can we just make robots do these things and, you know, difficult politically difficult to say, okay, well, if a chat robot can do it better, then yeah. Every single time that conversation came up and it became a small test chat robot failed. Hmm. So it was really interesting because conceptually, you know, automated chat was the whizzbang fancy look at all of this AI stuff, but it still never matched the quality of an actual person having a conversation.

Tim:
So I think I ran it twice that test, and it turns out real people are still real people, but if you don’t have chat at all, and it’s like, well, let’s set up a chat bot. It turns out we’re all used to the fact that chat bots are real now and chat bots take a lot to put all the effort into and to monitor. And at the end of your chat interaction, you’re still trying to have a lead come out, which is the same thing as filling out a form, but you’re making people talk to a robot first. So I sit in the camp of why make people go through all the extra hoops, because you’ve got a neat widget on your site. Don’t create a bunch of unnecessary steps for your users to get to a real person you’re creating this. I’m going to create automation so that they don’t have to talk to me. And at the end of the day, it’s the conversation that you’re actually going after. So don’t put a bunch of robots in the way of that conversation, a personal preference. I haven’t seen a chat robot win at an AB test, or even a longterm Legion test or B2B strategy over getting in front of real people, call me old school. But that’s just how it’s played out for me in the last several years.

Brian:
Yeah. Well, now you have the data to back it up. You literally pitted humans against robots. I’m very comforted to learn the humans one.

Tim:
Yeah. And I think we’ll continue to win.

Brian:
Great. Well on that very positive and encouraging note. Thanks so much. This, this has been really, really enlightening before we go. Where can people find you on the interwebs?

Tim:
Well, my name’s pretty easy. Seven characters, Tim Duke. And you can find me at https://mobile1st.com/. That’s mobile ‘1’ st.com. And yeah, that’s the agency where we focus on optimizing conversion from a mobile point of view because it turns out mobile devices aren’t going away.

Brian:
Perfect. Well, man, thanks again.

Tim:
Yeah, Brian really glad to have the conversation and dig up some old war stories. Hopefully it’s been helpful to anybody who’s listening.

© 2024 Brian David Hall