Hidden KPI: "Looking busy"
May 2019
Yesterday we looked at how Hidden KPIs can impact your team, testing program, and career. One of the most common examples is the unspoken expectation that everybody should always be doing … lots of stuff. This can undermine the success of a testing program, so let’s talk about how to deal with it.
Signs that “looking busy” is a hidden KPI for your organization
You probably already know if this is an issue, but in case you’re unsure, or just need validation, here are a few indicators:
- Time off. Does senior management take advantage of their vacation time, or are they somehow too swamped, or too enmeshed in the day-to-day, to take a week off?
- Letting go. Does the company regularly stop doing things - even profitable things - because they distract from its main mission? (Here’s an example of what that looks like - an oldie but a goodie.) Or does the number of activities and initiatives steadily grow over time?
- Meetings. Are meetings called with a clearly defined purpose, run with an agenda, attended only by those who need to be there, and concluded with concrete next steps? Or … not?
How this hidden KPI impacts your testing program
A busy experimentation program is not necessarily an effective one. In fact, sometimes you have to deliberately do nothing in order to be effective. So if a culture of busyness pushes you to run more tests, or run tests faster, you’re headed for disaster.
In particular, the push for more tests will likely force you into running lower impact tests on lower impact pages, elements, and audiences. You’ll certainly keep busy tracking all these concurrent experiments, but you won’t see much measurable return.
This leads to another danger, which is the risk that your reporting and results analysis take on a disproportionate role. A test has no value if we don’t draw conclusions and take action based on its results, so these are clearly important activities. But a culture of busyness can lead you to spend days diving into audience segments of insufficient sample size, increasing your risk of Type I errors. And it can lead you to produce extensive 30-slide results decks where a “here’s what we learned, here’s what we’re doing next” email might have sufficed.
What you can do
It’s hard to meet the Hidden KPI of looking busy while also hitting your actual KPIs of conversion lift or impact on revenue. Here are a few strategies that should help:
- Don’t test more. Let traffic, conversion rates, and knowledge about visitor behavior dictate where you focus your efforts. (If somebody wants to know why you’re not leveraging some ridiculous new feature of your testing tool on some trivial audience segment, just let them know you’ve got several higher impact tests in the pipeline, and you’ll look into that option next quarter.)
- Invest in your own development. Conduct an audit of statistical methods with an eye to reach significant results faster. Survey and try out testing tools to see if your team can save money and time. Research what others in your industry have done on their sites. This is the highest-leverage way you can stay busy.
- Make friends. You’re not the only one solving the problem of how to look busier than you actually are. Grab someone from another team and conduct a round of hallway usability testing on your funnel. You’ll learn something, they’ll learn something, and you’ll easily kill an hour. (Share what you’ve learned and it becomes several hours!)
- Market your team internally. Hold a brown bag lunch where you walk through what you’re testing, what you’ve learned in the last few months, and discuss challenges you’re facing.
If you’ve got your own tips and tricks for navigating a “look busy” culture, please reply and let me know!