👈 Home

Hidden KPI: "Looking busy"


Yesterday we looked at how Hidden KPIs can impact your team, testing program, and career. One of the most common examples is the unspoken expectation that everybody should always be doing … lots of stuff. This can undermine the success of a testing program, so let’s talk about how to deal with it.

Signs that “looking busy” is a hidden KPI for your organization

You probably already know if this is an issue, but in case you’re unsure, or just need validation, here are a few indicators:

How this hidden KPI impacts your testing program

A busy experimentation program is not necessarily an effective one. In fact, sometimes you have to deliberately do nothing in order to be effective. So if a culture of busyness pushes you to run more tests, or run tests faster, you’re headed for disaster.

In particular, the push for more tests will likely force you into running lower impact tests on lower impact pages, elements, and audiences. You’ll certainly keep busy tracking all these concurrent experiments, but you won’t see much measurable return.

This leads to another danger, which is the risk that your reporting and results analysis take on a disproportionate role. A test has no value if we don’t draw conclusions and take action based on its results, so these are clearly important activities. But a culture of busyness can lead you to spend days diving into audience segments of insufficient sample size, increasing your risk of Type I errors. And it can lead you to produce extensive 30-slide results decks where a “here’s what we learned, here’s what we’re doing next” email might have sufficed.

What you can do

It’s hard to meet the Hidden KPI of looking busy while also hitting your actual KPIs of conversion lift or impact on revenue. Here are a few strategies that should help:

If you’ve got your own tips and tricks for navigating a “look busy” culture, please reply and let me know!

© 2022 Brian David Hall